LE CODE EN COMMENTAIRE, C’EST JUSTE POUR POUVOIR KNIT PLUS VITE!! (avec cache = TRUE, ca re-run pas le code, mais le markdown doit quand meme compiler les graphes, du coup ca prend du temps à chq fois…)

names(cred) <- tolower(names(cred)) # changing the name of the variable into lowercase
cred <- cred[,-1] # deleting the first useless column
sum(is.na(cred)) # checking if we have missing data

Introduction

Parler de: - méthode utilisée - grandes étapes - classification task (goal) =/= prediction - [à compléter…]

–> Exploration des données = 1st insight –> Modelling: train/test set + selection du “meilleur” arbre(pruning)/svm(tuning)/NNET(nb of layers)/regression logicstic/K-NN(choisir K)/etc. + presentation de détail de tout ce bordel –> Cross validation avec les meilleurs de chq models pour selectionner le grand gagnant…

–> Description des variables (comme ca c’est plus simple de s’en sortir?!)

Exploratory analysis

Explanatory variables

Overview

Before beginning any kind of analysis, we have to understand the data we are working with.

data.frame(variable = names(cred),
           classe = sapply(cred, typeof),
           first_values = sapply(cred, 
                                 function(x) paste0(head(x),  collapse = ", ")),
           row.names = NULL) %>% 
  kable(caption="Overview of our data") %>% 
  kable_styling(bootstrap_options = c("striped", "hover", "condensed"), 
                full_width = F) %>%
  column_spec(1, width = "10em", border_right = T) %>%
  column_spec(2, width = "6em") %>%
  column_spec(3, width = "18em") %>%
  scroll_box(width = "65%", height = "250px")
Overview of our data
variable classe first_values
chk_acct integer 0, 1, 3, 0, 0, 3
duration integer 6, 48, 12, 42, 24, 36
history integer 4, 2, 4, 2, 3, 2
new_car integer 0, 0, 0, 0, 1, 0
used_car integer 0, 0, 0, 0, 0, 0
furniture integer 0, 0, 0, 1, 0, 0
radio.tv integer 1, 1, 0, 0, 0, 0
education integer 0, 0, 1, 0, 0, 1
retraining integer 0, 0, 0, 0, 0, 0
amount integer 1169, 5951, 2096, 7882, 4870, 9055
sav_acct integer 4, 0, 0, 0, 0, 4
employment integer 4, 2, 3, 3, 2, 2
install_rate integer 4, 2, 2, 2, 3, 2
male_div integer 0, 0, 0, 0, 0, 0
male_single integer 1, 0, 1, 1, 1, 1
male_mar_or_wid integer 0, 0, 0, 0, 0, 0
co.applicant integer 0, 0, 0, 0, 0, 0
guarantor integer 0, 0, 0, 1, 0, 0
present_resident integer 4, 2, 3, 4, 4, 4
real_estate integer 1, 1, 1, 0, 0, 0
prop_unkn_none integer 0, 0, 0, 0, 1, 1
age integer 67, 22, 49, 45, 53, 35
other_install integer 0, 0, 0, 0, 0, 0
rent integer 0, 0, 0, 0, 0, 0
own_res integer 1, 1, 1, 0, 0, 0
num_credits integer 2, 1, 1, 1, 2, 1
job integer 2, 2, 1, 2, 2, 1
num_dependents integer 1, 1, 2, 2, 2, 2
telephone integer 1, 0, 0, 0, 0, 1
foreign integer 0, 0, 0, 0, 0, 0
response integer 1, 0, 1, 1, 0, 1

 

As we can see, the data are coherent with the infos that “the client” provided us. Most of them are binary or categorical, while only few are numerical.

More than seeing the first values of our variables and their types, we also need to understand how distributed they are and their link with each other. Thanks to a correlation plot, we can see the correlation between each pair of variable, but especially their correlation with our response variable in which we are interested in.
We see for instances that variables like chk_acct, duration, history, sav_accnt or rent are highly correlated (positively or negatively) with our outcome variable and that they will be likeli to influe it in the models that we are going to plot. Others like present_resident or retaining should have low impact.

plot_correlation(cred, type="all", title = "Correlation Graph") # Attention! Il faut seulement ploter avec les valeurs continues. Les catégoriques se feront avec le khi-2! Test de wilkockson pour tester si les groupes sont différents (comme je le comprends, voir si une valeur cotinue a une influence sur l'outcome des groupes). Ensuite, fait des tests d'indépendance sur les discrètes pour voir si elles ont une influence sur les valeurs de réponse.

In addition, we can appreciate the summary of the different variables. The frequency table of history is presented below as an example:

Summary of variable history
Value
Min. 0.000
1st Qu. 2.000
Median 2.000
Mean 2.545
3rd Qu. 4.000
Max. 4.000
Frequency table of history
Values Frequency
0 40
1 49
2 530
3 88
4 293

 
 

However, presenting such a summary for all variables can be long and boring. It can be better to represent these number visually. A Boxplot is optimal to get all the important values for the numerical data, while a barplot will give us strong insights for categorical data. Let’s appreciate the following graphs:

for (i in 1:(length(cred)-1)) {
  if (range(cred[, i] < 5)) {
    print(
      ggplot(cred, aes(x = cred[, i])) +
        geom_bar(stat = "count", position = "dodge") +
        ggtitle(str_c("Barplot of\n", paste(
          colnames(cred[i])
        ))) +
        xlab(colnames(cred[i])) +
        ylab("Total") +
        my_theme()
    )
  } else
  {
    print(
      ggplot(cred, aes(y = cred[, i])) + geom_boxplot() +
        ylab(colnames(cred[i])) +
        ggtitle(str_c("Boxplot of\n ", paste(
          colnames(cred[i])
        ))) +
        my_theme() +
        theme(
          axis.text.x=element_blank())
    )
  }
}

Thanks to these graphs, we can better understand our data at a glance and will be able to refer to them when needed.

In addition, these graphs enable un too see that some data are not tidy. For instance, education should be a binary variable. However, we can see on the histogram of this variable that we have data where \(-1\) were recorded. We have the same problem for the binary variable guarantor were a value \(2\) is present.
In addition, we can also have strong suspitions that the variable age has wrong recorded data as we can see an outlier with a value much bigger than 100.
We will have to confirm our first assumptions and to modify these dirty data in an appropriate way.

Let’s first look at our variable age. We assume that, generally, a person will not live more than a hundred year, and will not contract a credit at such age. This is why the data with \(Age > 100\) are most likely wrongly recorded. We will therefore have to replace them in our database.
First, we have to find how much data are potentially dirty according to our assumptions and to localise them in order to replace them.

Number of instances with age > 100
Var1 Freq
FALSE 999
TRUE 1
Position of instance with age > 100
x
537

 

According to our results, we have one data with \(age > 100\) that has to be replaced. It is the instance 537 and its value is 125.

We can consider different options to replace this value. The first one could be to replace it by a value at random within the range (a value at random between 19 and 75, which is the second lowest value after \(125\).
However, according to the following histogram, the distribution of the age (without the erroneous data) is inequal with a concentration around small values (which is logical as young people generally have less money than elders and therefore are more subject to ask for credits).

It could therefore be possible to replace it at random with different probabilities according to the size of each class.
We prefer to opt for the median (equal to 33) to replace our problematic value as it offers more convenience.
Note that for calculating the median, our problematic value should not be used.

cred$age[which(age>75)] <- median(age[age!=max(age)])

An alternative could have been to use the mean, but, as we have no really big outlier, both values would have been close to eachother (\(mean = 35.596\) while \(median = 33\)).

 

Next, we also have to deal with our two categorical data that have been wrong recorded:
- one in education
- one in guarantor

They also have to be cleaned.

The following is again the barplot of education.

Here, the likelihood that this wrong recorded data is equal to \(0\) is clearly higher. Therefore, each of the previously presented methods (using the mean, using the median and even assigning it to a class at random) would with a high probability result in assigning this instance and assign it the value \(Education = 0\).
We can confirm these first assumption with a frequency table:

Frequency table of education
-1 0 1
sample size 1 950 49
proportion 0.1 % 95 % 4.9 %

 

It is indeed more appropriate to replace our value by 0 as the probability of belonging to this class is close to 20 times bigger.

cred$education[which(education == -1)] <- 0

 

Concerning the variable guarantor, we can look at the frequency table and plot the barplot as well:

Frequency table of guarantor
0 1 2
sample size 948 51 1
proportion 94.8 % 5.1 % 0.1 %

Again, for the same reasons, it is preferable to replace the wrong recorded data by 0.

cred$guarantor[which(guarantor == 2)] <- 0

In addition, the variable present_resident is also problematic as it doesn’t have the same range as the other categorical values. Its range goes from 1 to 4 whereas it should go from 0 to 3 like the other ones. We can modifiy its values in order to have the same format everywhere.

cred$present_resident <- subtract(cred$present_resident, 1)

These first steps have enabled us to better understand our explanatory variables and to clean the problematic ones.
We now have to focus in detail to the response variable on which the predictions should be made.

Response variable

As our final goal is to predict if a customer should be classified as a risky one or not, we have to have a particular look at our response variable that establishes if an applicant presents a good or a bad risk.
Let’s first have a look at its distribution:

Frequency table of our response variable
0 1
sample size 300 700
proportion 30 % 70 %

As we can see, if a random customer steps in the bank, the a priori probability that he will present a good ranking will be of 70%.
Without any calculations, the bank has more chances to make a good decision when octroying a credit.
However, the consequences can be really dramatic if 30% of the credits that the bank gives are not totally reimbursed. That’s why we have to develop a model to improve this initial accuracy that is obtained using a naive method of always octroying a credit.
Optimally, this model should also minimise the number of credits that are predicted as “good” and that are actually “bad” as the consequences for the bank (reimbursement of the credit by the customer) can be much more dramatic in this situation than in a situation where a “bad” credit is predicted as “good”.

We will get to this later. First, after having presented each variable, it could be interesting to see if we can already have some assumptions concerning the relations between the explanatory variables and the response variable.

Interactions between the explanatory variables and the response variable

Text…

–> Expliquer qu’on peut deja avoir un premier apriori sur les variables qui vont avoir un impact

** PAS TOP LES BOXPLOTS POUR LES VARIABLES BINAIRES ??!!**

Modelling

Before beginning to work on our different models, we can create a test set and a training set in order to build the models. This procedure is used is order to avoid overfitting and to predict instances which have been used while building the model.
We opt for a size of 70% of the original dataset for the training set and a size of 30% of the original dataset for the testing set.
In order to evalate the performance of the different models, we will have to use the exact same training and test sets for each model to be sure that the performance differences will result from the model we use and not from the randomess of splitting differently both sets.
A Cross-Validation will be performed at the end in order to compare the different models and to choose which one should be used to make good predictions.

# Creation of testing and training sets
set.seed(123) 
index.train <- sample(1:nrow(cred), size = nrow(cred) * 0.7, replace = FALSE)
cred.train <- cred[index.train,]
cred.test <- cred[-index.train,]

In addition, to evaluate our models, we will use the accuracy as main measure of performance. However, computing the accuracy for each model can be quite long… We prefer to build a function to be able to retrieve at any time based on a confusion matrix:

# Creating of a function to retrieve the accuracy from a confusion matrix
accuracy <- function(c){
  print(sum(diag(c)) / sum(c))
}

We can now begin to work on the different models that we will use in order to make our predictions.

CART

First, we begin our analysis by using a decision (classification) tree.

The goal of a decision tree is to predict the final class of our response variable (“good” or “bad”) by using a succession of binary rules to apply to our data.
Each node is created thanks to an algorithm that aims to minimize an impurity criterion. The feature and its underlying value that maximises the impurity reduction (that “best splits”the dataset in two) will be selected. This procedure is repeated until a stopping rule is reached.
At the end, we will have multiple branches that will all lead to the final forecast that we will make for a given instances.
Better than words, let’s compute the model and have a look at it.

Building the model

cart.model <- rpart(response ~ .,  data = cred.train, method = "class")
rpart.plot(cart.model, main = "Original decision tree")

–> At this time, we have obtained a decision tree that can be used to do the predictions. At it node, one should look at the values of the data and take the appropriate direction till arriving at the last row, when the prediction can be made.
However, this tree is also really complex: there are a lot of split and of branches.
We have to tune this initial model and make it more simple. We will simplify this model without loosing predictive capabilities by pruning it, keeping only the most important splits linked to the most important variables.

Tuning the model: Pruning

As already said, our complex tree has to be pruned in order to reduce its complexity. To do so, we will use the 1 - SE rule.
The idea of this rule is really general. As one would establish a t-test in statistics to see if two measures are statistically different, the 1 - SE rule tries to establish if two models produce statistically different results (if we can affirm that one outperforms the other).
We therefore consider the xerror (the criterion in which we are interested in and that we want to minimize) and its standard deviation xstd. A models that falls within 1 standard deviation of the most complex model can be considered as equivalent in term of performance. Therefore, using this rule, we will be able to prune the tree to get a much simplier model, without loosing in quality as the performance capability of our new model will be statistically the same as the first one.
To select the size of our pruned tree, we will look at the xerror of our original tree (the most complex one) and add one standard error xstd. We will then select the simplier tree with an xerror that lies within the calculated value.

Let’s have a look at these values from our original tree and decide where to prune it.

CP table of the CART model
CP nsplit rel error xerror xstd
0.0502392 0 1.0000000 1.0000000 0.0579320
0.0334928 2 0.8995215 1.0191388 0.0582450
0.0303030 3 0.8660287 1.0143541 0.0581678
0.0263158 6 0.7751196 1.0143541 0.0581678
0.0239234 8 0.7224880 0.9904306 0.0577715
0.0167464 9 0.6985646 0.9712919 0.0574422
0.0111643 11 0.6650718 0.9665072 0.0573581
0.0100000 14 0.6315789 0.9377990 0.0568392
Variable importance table first 6 instances
x
chk_acct 34.204041
duration 12.965308
history 11.651379
sav_acct 9.224824
amount 9.071985
employment 8.347383

Accoring to our first table, the minimal xerror is equal to 0.937799 , the minimal xstd to 0.0568392 and the sum of both is therefore equal to 0.9946382.

The smallest tree with an xerror below this value is equal to 0.9904306 and we will therefore prune the tree at CP = 0.0239234. According to our previous table, this represents 8 splits to be kept, equivalent to a tree of size 9.
It is also possible to observe this value on the graph where the dash line indicates the xerror plus its xstd from the most complex tree. We therefore select the less complex tree under this line, which is the tree of size 9 (or, again, with 8 splits).

We thus prune the tree at this value (the R function requires to indicate CP.)

cart.pruned <- prune(cart.model, cp = cp.pruned)

We can visualize this new pruned tree that is much smaller and therefore less complex than our original one and will use it to do our predictions, to build the confusion matrix and to calculate the underlying accuracy.

Predicting the values of the testing set

pred.cart <- predict(cart.pruned, newdata = cred.test, type = "class")

# confusion matrix:
cart.tab <- table(Predictions = pred.cart, Observations = cred.test$response) 
cart.tab %>% kable(caption = "Confusion matrix for the CART model",
                   col.names = c("Predict bad", "Predict good")) %>%
  kable_styling(
    bootstrap_options = c("striped", "hover", "condensed"),
    full_width = F,
    position = "l") %>%
  column_spec(1, border_right = T, width = "5em") %>%
  column_spec(2, width = "6em") %>%
  column_spec(3, width = "6em")
Confusion matrix for the CART model
Predict bad Predict good
bad 36 27
good 55 182

Based on this table, we can calculate the accuracy using our previously build function, and that calculates the element weel classified divided by the total number of elements.

# accuracy using our previously build function
accuracy(cart.tab) 
## [1] 0.7266667

It is possible to have more informaion, with for instance the sum of each rows with the following cross table.


 
   Cell Contents
|-------------------------|
|                       N |
|           N / Row Total |
|           N / Col Total |
|         N / Table Total |
|-------------------------|

 
Total Observations in Table:  300 

 
                   | pred.cart 
cred.test$response |       bad |      good | Row Total | 
-------------------|-----------|-----------|-----------|
               bad |        36 |        55 |        91 | 
                   |     0.396 |     0.604 |     0.303 | 
                   |     0.571 |     0.232 |           | 
                   |     0.120 |     0.183 |           | 
-------------------|-----------|-----------|-----------|
              good |        27 |       182 |       209 | 
                   |     0.129 |     0.871 |     0.697 | 
                   |     0.429 |     0.768 |           | 
                   |     0.090 |     0.607 |           | 
-------------------|-----------|-----------|-----------|
      Column Total |        63 |       237 |       300 | 
                   |     0.210 |     0.790 |           | 
-------------------|-----------|-----------|-----------|

We can see that 91 are predicated as bad and 209 are predicted as good.

–> Parler de accuracy en elle meme (0.72 pas top par rapport aux 0.7 de base si on prédit que “good”) –> Parler des bonnes predictions de 0 (c’est elles qui nous intéressent et prédire que 1 conduirait la banque à faire faillite) –> Parler FALSE POSTIVIE / FALSE NEGATIVE / TRUE POSITIVE / TRUE NEGATIVE (OU A LA FIN)

Neural Network

Building the model and optimizing it: selecting the number of neurones in the hidden layer

A MODIFIER // SUPPRIMER La “littérature” (https://stats.stackexchange.com/questions/181/how-to-choose-the-number-of-hidden-layers-and-nodes-in-a-feedforward-neural-netw) suggère de garder 1 seul layer un nb de neuronnes entre 1 et le nbr de dim: on va les tester tous et garder le meilleur… “In sum, for most problems, one could probably get decent performance (even without a second optimization step) by setting the hidden layer configuration using just two rules: (i) number of hidden layers equals one; and (ii) the number of neurons in that layer is the mean of the neurons in the input and output layers.”

“Over-parametrization quickly lead to instability in the estimate and can lead to overfitting. One possibility is to impose a penalty on the largest weights during the optimization. This is called regularization and, more specifically in the context of neural network, weight decay.”

–> On essaye de trouver le nombre “optimal” de neurones. Vu que neural network utilise des valeurs aléatoires au début pour lancer l’algo etc. Ca varie. On calcule donc 5 fois l’accuracy liée à chaque nombre de neurones et voir si y’a un nb de neurones optimal.

train_control <- trainControl(method = "cv", number = 5)

nnet_fit <- train(form = response~ .,
                 data = cred,
                 trControl = train_control,
                 tuneGrid = expand.grid(size = 14:25, decay = c(1.0, 1.5, 2)),
                 method = "nnet")

plot(nnet_fit)

nnet_fit$results
nnet_fit$results[which.max(nnet_fit$results$Accuracy),]$size # the size maximizing the accuracy
nnet_fit$results[which.max(nnet_fit$results$Accuracy),]$decay # the decay maximizing the accuracy

Model selection and predicting the values of the testing set

We present you the characteristics of the model retained, with 24 neurones in the hidden layer and a decay of 2. As usual, we trained the model and predict the values of the test set to be able to build the confusion matrix and to calculate the accuracy.

nnet.model.retained <- 
  nnet(cred.train$response ~ .,
       data = cred.train, maxit = 200,
       size = nnet_fit$results[which.max(nnet_fit$results$Accuracy),]$size, 
       decay = nnet_fit$results[which.max(nnet_fit$results$Accuracy),]$decay)

# Predictions on the test set:
pred.nnet.retained <- predict(nnet.model.retained, cred.test, type="class") 

# Confusion matrix
tab.nnet.retained <- table(Reality = cred.test$response, 
                           Predicted = unlist(pred.nnet.retained)) 
# Accuracy
acc.nnet.retained <- sum(ifelse(
  cred.test$response == unlist(pred.nnet.retained), 1, 0), na.rm = TRUE) /
  length(cred.test$response)

In addition, we can plot our neural network:

This graph shows that our model can be quite complex, there a plenty of arrows. It is not necessary to understand precisely all of them and they can be seen as a “black box”, meaning that the interpretability of such a model is modest. But, afterall, what we need is to make good predictions!

Confusion matrix of the fitted NNET model
Predict good Predict bad
bad 46 45
good 17 192

We obtain a final accuracy of 0.793 on our testing set. Note that this accuracy is different from the one using the cross validation before as here, we use only one test set that may be disproportioned. We will, at the end, use again a cross validation for all models on the same train and test sets in order to compare the different models.

Support Vector Machine

Another important model that we will now consider and compare to the other ones will be the Support Vector Machines.
This models aims at… DESCRIPTION

Building the model

Again, we decide to use the caret package to be able to compare models with different costs. We will then select the best performing model and analyse it separately.

svm_parameters <- data.frame(C = seq(0.5, 5, 0.5)) 
# note that we first tested a wider scope of values (from 0 to 1000)
# and now present only the interval that leads to the best results

svm_fit <- train(form = response ~ .,
                   data = cred,
                   method = "svmRadialCost",
                   tuneGrid = svm_parameters,
                   preProcess = "range",
                   trace = FALSE,
                   trControl = train_control)

Optimizing the model: selecting good cost parameter

Now that our different SVM models with different cost parameters have been built, we have to select the cost parameter that leads to the highest accuracy.

plot(svm_fit)

svm_fit$results
##      C Accuracy     Kappa AccuracySD    KappaSD
## 1  0.5    0.755 0.3083141 0.01581139 0.06156416
## 2  1.0    0.766 0.3791768 0.02837252 0.07072015
## 3  1.5    0.765 0.3837991 0.03041381 0.08704831
## 4  2.0    0.755 0.3687115 0.03588175 0.09494762
## 5  2.5    0.753 0.3649070 0.03510698 0.09285237
## 6  3.0    0.753 0.3688597 0.03581201 0.09338160
## 7  3.5    0.755 0.3773095 0.03482097 0.09325352
## 8  4.0    0.761 0.3927647 0.03267262 0.08597879
## 9  4.5    0.761 0.3923427 0.03489269 0.09513774
## 10 5.0    0.761 0.3922295 0.03070016 0.08670766
svm_fit$results[which.max(svm_fit$results$Accuracy),]$C # the cost maximizing the accuracy
## [1] 1

According to our results and by having tested different costs, we conclude that we should opt for a cost of 1 which leads to the highest accuracy.

We can then “extract” this model and analyze it on his own. We have to note that, as the Neural Network, SVM doesn’t allow for much interpretations. What matters here is the quality of the predictions more than the individual interpretability of the different variables.
In addition, to have a 2D graph, one should fix (slice) all other variables. Here, having 30 explanative variables, it does not make sense to fix 28 out of them and plotting is therefore not appropriate.

However, we can look at the predictions of the model.

Predicting the values of the testing set

svm.model.retained <- 
  svm(cred.train$response ~ .,
       data = cred.train, 
       cost = svm_fit$results[which.max(svm_fit$results$Accuracy),]$C)

# Predictions on the test set:
pred.svm.retained <- predict(nnet.model.retained, cred.test, type="class") 

# Confusion matrix
tab.svm.retained <- table(Reality = cred.test$response, 
                           Predicted = unlist(pred.svm.retained)) 
# Accuracy
acc.svm.retained <- accuracy(tab.nnet.retained)
## [1] 0.7933333

We can build the confusion matrix and see how the models performs.

tab.svm.retained %>% kable(
  caption = "Confusion matrix of the fitted SVM model",
  col.names = c("Predict good", "Predict bad")) %>%
  kable_styling(
    bootstrap_options = c("striped", "hover", "condensed"),
    full_width = F,
    position = "l"
  ) %>%
  column_spec(1, border_right = T, width = "5em") %>%
  column_spec(2, width = "6em") %>%
  column_spec(3, width = "6em")
Confusion matrix of the fitted SVM model
Predict good Predict bad
bad 46 45
good 17 192

With this model we obtain a final accuracy of 0.7933333 on the original testing set that we built at the beginning (without CV).
Again, we will compare this model with the others in a final step using a cross validation procedure performed on the same training sets and testing sets at the end of our analysis.

Random Forest

Another model that we will consider is the Random Forest. Considering its construction, the Random Forest is a really special model.
Indeed, the model is built DESCRIPTION….

Using our caret package again as our most powerful tool enabling to build an easy and readable code, we will test different mtry which correspond to number of variables that we consider at each split of the tree. Therefore, at each split, we consider only some variables (while in a regular CART model, we consider all of them), what will lead the different trees to be uncorrelated and increase their prediction power.
** COMPLETER UN PEU…**

Finally, we will select the mtry that leads to the highest accuracy.

rf_parameters <- data.frame(mtry = c(7:14))
# note that we first tested a wider scope of values (from 0 to 1000)
# and now present only the interval that leads to the best results

rf_fit <- train(form = response ~ .,
                 data = cred,
                 method = "rf",
                 ntree = 500, 
                 tuneGrid = rf_parameters,
                 preProcess = "range",
                 trace = FALSE,
                 trControl = train_control)

We see that the error rate remains really similar past a certain value (around 100), meaning that increasing the number of tree does not allow to increase the predictive capabilities anymore. The second derivative is negative and the decrease of the error rate becomes smaller and smaller the more trees we add. We therefore decide to stick to this initial value of 500 trees.

Now that we have built our model, we can select the optimal mtry to continue our analysis.

plot(rf_fit) 

rf_fit$results
##   mtry Accuracy     Kappa  AccuracySD     KappaSD
## 1    7    0.763 0.3767828 0.004472136 0.018824784
## 2    8    0.767 0.3836047 0.007582875 0.019358916
## 3    9    0.758 0.3686510 0.013038405 0.028501315
## 4   10    0.753 0.3559238 0.008366600 0.026932788
## 5   11    0.764 0.3892564 0.010246951 0.024961649
## 6   12    0.757 0.3704659 0.004472136 0.009564577
## 7   13    0.755 0.3665806 0.009354143 0.024351521
## 8   14    0.756 0.3709787 0.008215838 0.025203608
rf_fit$results[which.max(rf_fit$results$Accuracy),]$mtry # the cost maximizing the accuracy
## [1] 8

According to these results, we see that we should consider 8 variables at each split, which leads to the highest accuracy.

We can now analyse more in depth this model.

rf.model.retained <- 
  randomForest(formula = response ~ .,
               data = cred.train, 
               ntree = 500, 
               mtry = rf_fit$results[which.max(svm_fit$results$Accuracy),]$mtry, 
               importance = TRUE)

Note that we decide not to make vary ntree, the number of trees used in the Random Forest and this number has been set to 500. This is the default number and increasing it more would have a quasi null impact. We can verify this assumption thanks to the following graph.

plot(rf.model.retained)

Now that we have built our final model, contrarily to the regular CART, we can not directly see which variables are the most important looking at the branches of each tree.
However, not like for the Neural Network or SVM, it is still possible to see the individual contribution of each variables thanks to the variable importance plot.

INTERPRETATION DE COMMENT CA MARCHE –> on retire un variable ca diminue –> variables importantes en haut

varImpPlot(rf.model.retained)

Finally, we can predict the values of our testing set, build the confusion matrix and predict the accuracy.

## [1] 0.77
tab.rf.retained %>% kable(
  caption = "Confusion matrix of the fitted SVM model",
  col.names = c("Predict good", "Predict bad")) %>%
  kable_styling(
    bootstrap_options = c("striped", "hover", "condensed"),
    full_width = F,
    position = "l"
  ) %>%
  column_spec(1, border_right = T, width = "5em") %>%
  column_spec(2, width = "6em") %>%
  column_spec(3, width = "6em")
Confusion matrix of the fitted SVM model
Predict good Predict bad
bad 31 60
good 9 200

Using the Random Forest, we obtain a final accuracy of 0.77 on the original testing set.

K - Nearest Neighbors

Building the model and Optimizing the model: selecting the number of neighbors and the distance measure

The following code will not be run each time we execute our evaluation of the different models in order to gain in computing efficiency.
However, we ran it one to be able to evaluate which distance measure to select.

kknn_parameters <- expand.grid(k = 2:30, distance = 1:5, kernel= "optimal")

kknn_fit <- train(form = response~ .,
                 data = cred,
                 trControl = train_control,
                 tuneGrid = knn_parameters,
                 method = "kknn",
                 preProcess = c("center", "scale"))
plot(kknn_fit)
kknn_fit$results

kknn_fit$results[which.max(knn_fit$results$Accuracy),]$k
kknn_fit$results[which.max(knn_fit$results$Accuracy),]$distance

After having created multiple K-Nearest Neighbors models with the knn function, we realize that the distance of 2 outperforms other distances. We can therefore use the function knn to illustrate how our final model is built and how we select k (note that knn function is more efficient in term of computationnal power, only reason why we use it to choose k rather than the kknn function just above.)

We can thus use the knn function to see how the accuracy varies with different k between 2 an 30 and select the number of neighbors that lead to the highest accuracy.

knn_parameters <- expand.grid(k = 2:30)

knn_fit <- train(form = response~ .,
                 data = cred,
                 trControl = train_control,
                 tuneGrid = knn_parameters,
                 method = "knn",
                 preProcess = c("center", "scale"))
plot(knn_fit)

knn_fit$results
##     k Accuracy     Kappa  AccuracySD    KappaSD
## 1   2    0.672 0.2127481 0.019557607 0.04544684
## 2   3    0.731 0.3145823 0.013874437 0.02813746
## 3   4    0.731 0.3108099 0.035951356 0.09758925
## 4   5    0.733 0.3052926 0.017535678 0.04274584
## 5   6    0.726 0.2817805 0.023291629 0.06306552
## 6   7    0.730 0.2810081 0.025248762 0.05260888
## 7   8    0.733 0.2870455 0.017175564 0.04609015
## 8   9    0.727 0.2641373 0.037682887 0.09336581
## 9  10    0.721 0.2413165 0.022192341 0.05339896
## 10 11    0.735 0.2711707 0.013693064 0.02943665
## 11 12    0.735 0.2700694 0.021794495 0.04539353
## 12 13    0.743 0.2894554 0.026598872 0.05538880
## 13 14    0.735 0.2602196 0.023717082 0.05449739
## 14 15    0.742 0.2714769 0.029706902 0.06348620
## 15 16    0.742 0.2709635 0.027064737 0.06298523
## 16 17    0.728 0.2188597 0.023611438 0.04858919
## 17 18    0.737 0.2434800 0.020796634 0.05164192
## 18 19    0.738 0.2374734 0.016046807 0.05285351
## 19 20    0.727 0.2050274 0.017175564 0.04972431
## 20 21    0.728 0.2038275 0.018907670 0.04586525
## 21 22    0.723 0.1881667 0.017888544 0.04606450
## 22 23    0.725 0.1843142 0.015411035 0.04148163
## 23 24    0.720 0.1712408 0.024748737 0.06010552
## 24 25    0.730 0.1925921 0.018027756 0.05141921
## 25 26    0.731 0.1946736 0.013416408 0.03385770
## 26 27    0.732 0.1945566 0.015652476 0.04818391
## 27 28    0.727 0.1819060 0.015247951 0.05668436
## 28 29    0.726 0.1682095 0.010839742 0.04180819
## 29 30    0.722 0.1606283 0.009746794 0.02649324
knn_fit$results[which.max(knn_fit$results$Accuracy),]$k
## [1] 13

The selected knn model uses a distance of 2 and is computed with the 13 nearest neighbors. The obtained accuracy using a 5 fold Cross-Validation is equal to 0.743.

Predicting the values of the testing set

Confusion matrix for the K-NN model
Predict bad Predict good
bad 36 27
good 55 182

Given our confusion matrix, on our initial testing set, the obtained accuracy is equal to .

Cross Validation approach

At this point, let’s make a summary of the situation.
Until now, we have built several different models and we have tuned them (selected their parameters), so the different models that we have are the best SVM, the best Neural Network etc.
However, we still have to confront each model to be able to select the overall winner.
In order to be able to compare them, we will not evaluate them only on a testing as evaluating the performance of the model once will not enable us to make any conclusions. For instance, if the concerned testing set is disproportionned, we could attribute the good score of a model to the predictive capabilities of the model while it comes in fact from the test set.
To be sure that the score of the model are not impacted (too much) by the characteristics of the testing set, we will perform a 10-fold Cross Validation.
With this technique, we will use 10 different training set and 10 different testing sets. Our original dataset will be split in 10, using at each iteration 1/10 of the original dataset as a testing set and the rest as a training set. Using this method, each instance will be once (and exactly once) in the testing set.
We will then be able to compute the accuracy on the different testing set for each iteration of the cross validation and then evaluate the average accuracy which will represent the numbers that we can expect the model to classify correctly.

We will firstly make the whole procedure “manually” as an example and then use the famous caret package again to make the code as simple as possible.

Thus, we begin with a creation of 10 different sets in order to do the Cross-Validation.

The number of instances being 1000, we will make 10 sets of size 100. They are stored in a list named test.list. At each step, the remaining part of the data base is stored in the list train.test.

test.list <- list() # creates an empty list that will be the test sets
train.list <- list() # creates an empty list that will be the train sets
counter <- 0

# Creates the 10 sets of size 300
for (i in 1:10){
  index <- counter + c(1:100) # the row numbers that will be in the test set
  test.list[[i]] <- cred[index, ] # the test set number i
  train.list[[i]] <- cred[-index, ] # the train set number i
  counter <- counter + 100
}

For example, test.list[[1]] is a data set of 300 rows taken at random from our data.

test.list[[1]] %>% kable(caption="Our first test set named test.list[[1]]") %>%
  kable_styling(bootstrap_options = c("striped", "hover", "condensed"),
                full_width = F) %>%
  scroll_box(width = "100%", height = "200px")
Our first test set named test.list[[1]]
chk_acct duration history new_car used_car furniture radio.tv education retraining amount sav_acct employment install_rate male_div male_single male_mar_or_wid co.applicant guarantor present_resident real_estate prop_unkn_none age other_install rent own_res num_credits job num_dependents telephone foreign response
0 6 4 0 0 0 1 0 0 1169 4 4 4 0 1 0 0 0 3 1 0 67 0 0 1 2 2 1 1 0 good
1 48 2 0 0 0 1 0 0 5951 0 2 2 0 0 0 0 0 1 1 0 22 0 0 1 1 2 1 0 0 bad
3 12 4 0 0 0 0 1 0 2096 0 3 2 0 1 0 0 0 2 1 0 49 0 0 1 1 1 2 0 0 good
0 42 2 0 0 1 0 0 0 7882 0 3 2 0 1 0 0 1 3 0 0 45 0 0 0 1 2 2 0 0 good
0 24 3 1 0 0 0 0 0 4870 0 2 3 0 1 0 0 0 3 0 1 53 0 0 0 2 2 2 0 0 bad
3 36 2 0 0 0 0 1 0 9055 4 2 2 0 1 0 0 0 3 0 1 35 0 0 0 1 1 2 1 0 good
3 24 2 0 0 1 0 0 0 2835 2 4 3 0 1 0 0 0 3 0 0 53 0 0 1 1 2 1 0 0 good
1 36 2 0 1 0 0 0 0 6948 0 2 2 0 1 0 0 0 1 0 0 35 0 1 0 1 3 1 1 0 good
3 12 2 0 0 0 1 0 0 3059 3 3 2 1 0 0 0 0 3 1 0 61 0 0 1 1 1 1 0 0 good
1 30 4 1 0 0 0 0 0 5234 0 0 4 0 0 1 0 0 1 0 0 28 0 0 1 2 3 1 0 0 bad
1 12 2 1 0 0 0 0 0 1295 0 1 3 0 0 0 0 0 0 0 0 25 0 1 0 1 2 1 0 0 bad
0 48 2 0 0 0 0 0 1 4308 0 1 3 0 0 0 0 0 3 0 0 24 0 1 0 1 2 1 0 0 bad
1 12 2 0 0 0 1 0 0 1567 0 2 1 0 0 0 0 0 0 0 0 22 0 0 1 1 2 1 1 0 good
0 24 4 1 0 0 0 0 0 1199 0 4 4 0 1 0 0 0 3 0 0 60 0 0 1 2 1 1 0 0 bad
0 15 2 1 0 0 0 0 0 1403 0 2 2 0 0 0 0 0 3 0 0 28 0 1 0 1 2 1 0 0 good
0 24 2 0 0 0 1 0 0 1282 1 2 4 0 0 0 0 0 1 0 0 32 0 0 1 1 1 1 0 0 bad
3 24 4 0 0 0 1 0 0 2424 4 4 4 0 1 0 0 0 3 0 0 53 0 0 1 2 2 1 0 0 good
0 30 0 0 0 0 0 0 1 8072 4 1 2 0 1 0 0 0 2 0 0 25 1 0 1 3 2 1 0 0 good
1 24 2 0 1 0 0 0 0 12579 0 4 4 0 0 0 0 0 1 0 1 44 0 0 0 1 3 1 1 0 bad
3 24 2 0 0 0 1 0 0 3430 2 4 3 0 1 0 0 0 1 0 0 31 0 0 1 1 2 2 1 0 good
3 9 4 1 0 0 0 0 0 2134 0 2 4 0 1 0 0 0 3 0 0 48 0 0 1 3 2 1 1 0 good
0 6 2 0 0 0 1 0 0 2647 2 2 2 0 1 0 0 0 2 1 0 44 0 1 0 1 2 2 0 0 good
0 10 4 1 0 0 0 0 0 2241 0 1 1 0 1 0 0 0 2 1 0 48 0 1 0 2 1 2 0 1 good
1 12 4 0 1 0 0 0 0 1804 1 1 3 0 1 0 0 0 3 0 0 44 0 0 1 1 2 1 0 0 good
3 10 4 0 0 1 0 0 0 2069 4 2 2 0 0 1 0 0 0 0 0 26 0 0 1 2 2 1 0 1 good
0 6 2 0 0 1 0 0 0 1374 0 2 1 0 1 0 0 0 1 1 0 36 1 0 1 1 1 1 1 0 good
3 6 0 0 0 0 1 0 0 426 0 4 4 0 0 1 0 0 3 0 0 39 0 0 1 1 1 1 0 0 good
2 12 1 0 0 0 1 0 0 409 3 2 3 0 0 0 0 0 2 1 0 42 0 1 0 2 2 1 0 0 good
1 7 2 0 0 0 1 0 0 2415 0 2 3 0 1 0 0 1 1 1 0 34 0 0 1 1 2 1 0 0 good
0 60 3 0 0 0 0 0 1 6836 0 4 3 0 1 0 0 0 3 0 1 63 0 0 1 2 2 1 1 0 bad
1 18 2 0 0 0 0 0 1 1913 3 1 3 0 0 1 0 0 2 1 0 36 1 0 1 1 2 1 1 0 good
0 24 2 0 0 1 0 0 0 4020 0 2 2 0 1 0 0 0 1 0 0 27 1 0 1 1 2 1 0 0 good
1 18 2 1 0 0 0 0 0 5866 1 2 2 0 1 0 0 0 1 0 0 30 0 0 1 2 2 1 1 0 good
3 12 4 0 0 0 0 0 1 1264 4 4 4 0 1 0 0 0 3 0 1 57 0 1 0 1 1 1 0 0 good
2 12 2 0 0 1 0 0 0 1474 0 1 4 0 0 0 0 0 0 0 0 33 1 0 1 1 3 1 1 0 good
1 45 4 0 0 0 1 0 0 4746 0 1 4 0 1 0 0 0 1 0 0 25 0 0 1 2 1 1 0 0 bad
3 48 4 0 0 0 0 0 0 6110 0 2 1 0 1 0 0 0 2 0 1 31 1 0 0 1 2 1 1 0 good
2 18 2 0 0 0 1 0 0 2100 0 2 4 0 1 0 1 0 1 1 0 37 1 0 1 1 2 1 0 0 bad
2 10 2 0 0 0 0 0 0 1225 0 2 2 0 1 0 0 0 1 0 0 37 0 0 1 1 2 1 1 0 good
1 9 2 0 0 0 1 0 0 458 0 2 4 0 1 0 0 0 2 1 0 24 0 0 1 1 2 1 0 0 good
3 30 2 0 0 0 1 0 0 2333 2 4 4 0 1 0 0 0 1 0 0 30 1 0 1 1 3 1 0 0 good
1 12 2 0 0 0 1 0 0 1158 2 2 3 1 0 0 0 0 0 0 0 26 0 0 1 1 2 1 1 0 good
1 18 3 0 0 0 0 0 0 6204 0 2 2 0 1 0 0 0 3 1 0 44 0 0 1 1 1 2 1 0 good
0 30 4 0 1 0 0 0 0 6187 1 3 1 0 0 1 0 0 3 0 0 24 0 1 0 2 2 1 0 0 good
0 48 4 0 1 0 0 0 0 6143 0 4 4 0 0 0 0 0 3 0 1 58 1 0 0 2 1 1 0 0 bad
3 11 4 1 0 0 0 0 0 1393 0 1 4 0 0 0 0 0 3 0 0 35 0 0 1 2 3 1 0 0 good
3 36 2 0 0 0 1 0 0 2299 2 4 4 0 1 0 0 0 3 0 0 39 0 0 1 1 2 1 0 0 good
0 6 2 0 1 0 0 0 0 1352 2 0 1 0 0 0 0 0 1 0 0 23 0 1 0 1 0 1 1 0 good
3 11 4 1 0 0 0 0 0 7228 0 2 1 0 1 0 0 0 3 0 0 39 0 0 1 2 1 1 0 0 good
3 12 2 0 0 0 1 0 0 2073 1 2 4 0 0 0 1 0 1 1 0 28 0 0 1 1 2 1 0 0 good
1 24 3 0 0 1 0 0 0 2333 4 1 4 0 1 0 0 0 1 0 0 29 1 0 1 1 1 1 0 0 good
1 27 3 0 1 0 0 0 0 5965 0 4 1 0 1 0 0 0 1 0 0 30 0 0 1 2 3 1 1 0 good
3 12 2 0 0 0 1 0 0 1262 0 2 3 0 1 0 0 0 1 0 0 25 0 0 1 1 2 1 0 0 good
3 18 2 0 1 0 0 0 0 3378 4 2 2 0 1 0 0 0 0 0 0 31 0 0 1 1 2 1 1 0 good
1 36 3 1 0 0 0 0 0 2225 0 4 4 0 1 0 0 0 3 0 1 57 1 0 0 2 2 1 1 0 bad
3 6 1 1 0 0 0 0 0 783 4 2 1 0 1 0 0 1 1 1 0 26 1 0 1 1 1 2 0 0 good
1 12 2 0 0 0 1 0 0 6468 4 0 2 0 1 0 0 0 0 0 1 52 0 0 1 1 3 1 1 0 bad
3 36 4 0 0 0 1 0 0 9566 0 2 2 0 0 0 0 0 1 0 0 31 1 0 1 2 2 1 0 0 good
2 18 2 1 0 0 0 0 0 1961 0 4 3 0 0 0 0 0 1 0 0 23 0 0 1 1 3 1 0 0 good
0 36 4 0 0 1 0 0 0 6229 0 1 4 0 0 0 1 0 3 0 1 23 0 1 0 2 1 1 1 0 bad
1 9 2 0 0 0 0 0 1 1391 0 2 2 0 0 1 0 0 0 1 0 27 1 0 1 1 2 1 1 0 good
1 15 4 0 0 0 1 0 0 1537 4 4 4 0 1 0 0 1 3 1 0 50 0 0 1 2 2 1 1 0 good
1 36 0 0 0 0 0 0 1 1953 0 4 4 0 1 0 0 0 3 0 1 61 0 0 0 1 3 1 1 0 bad
1 48 0 0 0 0 0 0 1 14421 0 2 2 0 1 0 0 0 1 0 0 25 0 0 1 1 2 1 1 0 bad
3 24 2 0 0 0 1 0 0 3181 0 1 4 0 0 0 0 0 3 0 0 26 0 0 1 1 2 1 1 0 good
3 27 2 0 0 0 0 0 0 5190 4 4 4 0 1 0 0 0 3 0 0 48 0 0 1 4 2 2 1 0 good
3 12 2 0 0 0 1 0 0 2171 0 1 2 0 0 0 0 0 1 0 0 29 1 0 1 1 2 1 0 0 good
1 12 2 1 0 0 0 0 0 1007 3 2 4 0 0 1 0 0 0 1 0 22 0 0 1 1 2 1 0 0 good
3 36 2 0 0 0 0 1 0 1819 0 2 4 0 1 0 0 0 3 0 1 37 1 0 0 1 2 1 1 0 bad
3 36 2 0 0 0 1 0 0 2394 4 2 4 0 0 0 0 0 3 0 0 25 0 0 1 1 2 1 0 0 good
3 36 2 0 1 0 0 0 0 8133 0 2 1 0 0 0 0 0 1 0 0 30 1 0 1 1 2 1 0 0 good
3 7 4 0 0 0 1 0 0 730 4 4 4 0 1 0 0 0 1 0 0 46 0 1 0 2 1 1 1 0 good
0 8 4 0 0 0 0 0 0 1164 0 4 3 0 1 0 0 0 3 0 1 51 1 0 0 2 3 2 1 0 good
1 42 4 0 0 0 0 0 1 5954 0 3 2 0 0 0 0 0 0 1 0 41 1 0 1 2 1 1 0 0 good
0 36 2 0 0 0 0 1 0 1977 4 4 4 0 1 0 0 0 3 0 1 40 0 0 1 1 3 1 1 0 bad
0 12 4 0 1 0 0 0 0 1526 0 4 4 0 1 0 0 0 3 0 1 66 0 0 0 2 3 1 0 0 good
0 42 2 0 0 0 1 0 0 3965 0 1 4 0 1 0 0 0 2 0 0 34 0 0 1 1 2 1 0 0 bad
1 11 3 0 0 0 1 0 0 4771 0 3 2 0 1 0 0 0 3 0 0 51 0 0 1 1 2 1 0 0 good
3 54 0 0 1 0 0 0 0 9436 4 2 2 0 1 0 0 0 1 0 0 39 0 0 1 1 1 2 0 0 good
1 30 2 0 0 1 0 0 0 3832 0 1 2 0 0 1 0 0 0 0 0 22 0 0 1 1 2 1 0 0 good
3 24 2 0 0 0 1 0 0 5943 4 1 1 0 0 0 0 0 0 0 0 44 0 0 1 2 2 1 1 0 bad
3 15 2 0 0 0 1 0 0 1213 2 4 4 0 1 0 0 0 2 0 0 47 1 0 1 1 2 1 1 0 good
3 18 2 0 0 0 0 0 1 1568 1 2 3 0 0 0 0 0 3 0 0 24 0 1 0 1 1 1 0 0 good
0 24 2 0 0 0 0 0 0 1755 0 4 4 0 0 0 0 1 3 1 0 58 0 0 1 1 1 1 1 0 good
0 10 2 0 0 0 1 0 0 2315 0 4 3 0 1 0 0 0 3 1 0 52 0 0 1 1 1 1 0 0 good
3 12 4 0 0 0 0 0 1 1412 0 2 4 0 0 0 0 1 1 1 0 29 0 0 1 2 3 1 1 0 good
1 18 4 0 0 1 0 0 0 1295 0 1 4 0 0 0 0 0 0 0 0 27 0 0 1 2 2 1 0 0 good
1 36 2 0 0 0 0 1 0 12612 1 2 1 0 1 0 0 0 3 0 1 47 0 0 0 1 2 2 1 0 bad
0 18 2 1 0 0 0 0 0 2249 1 3 4 0 1 0 0 0 2 0 0 30 0 0 1 1 3 2 1 0 good
0 12 0 0 0 0 0 0 0 1108 0 3 4 0 1 0 0 0 2 1 0 28 0 0 1 2 2 1 0 0 bad
3 12 4 0 0 0 1 0 0 618 0 4 4 0 1 0 0 0 3 1 0 56 0 0 1 1 2 1 0 0 good
0 12 4 0 1 0 0 0 0 1409 0 4 4 0 1 0 0 0 2 1 0 54 0 0 1 1 2 1 0 0 good
3 12 4 0 0 0 1 0 0 797 4 4 4 0 0 0 0 0 2 0 0 33 1 0 1 1 1 2 0 0 bad
2 24 4 0 0 1 0 0 0 3617 4 4 4 0 1 0 1 0 3 0 1 20 0 1 0 2 2 1 0 0 good
1 12 2 1 0 0 0 0 0 1318 3 4 4 0 1 0 0 0 3 1 0 54 0 0 1 1 2 1 1 0 good
1 54 0 0 0 0 0 0 1 15945 0 1 3 0 1 0 0 0 3 0 1 58 0 1 0 1 2 1 1 0 bad
3 12 4 0 0 0 0 1 0 2012 4 3 4 0 0 0 0 0 1 0 0 61 0 0 1 1 2 1 0 0 good
1 18 2 0 0 0 0 0 1 2622 1 2 4 0 1 0 0 0 3 0 0 34 0 0 1 1 2 1 0 0 good
1 36 4 0 0 0 1 0 0 2337 0 4 4 0 1 0 0 0 3 1 0 36 0 0 1 1 2 1 0 0 good
1 20 3 0 1 0 0 0 0 7057 4 3 3 0 1 0 0 0 3 0 0 36 1 1 0 2 3 2 1 0 good 
train.list[[1]] %>% kable(caption="Train set associated with test.list[[1]]") %>%
  kable_styling(bootstrap_options = c("striped", "hover", "condensed"),
                full_width = F) %>%
  scroll_box(width = "100%", height = "200px")
Train set associated with test.list[[1]]
chk_acct duration history new_car used_car furniture radio.tv education retraining amount sav_acct employment install_rate male_div male_single male_mar_or_wid co.applicant guarantor present_resident real_estate prop_unkn_none age other_install rent own_res num_credits job num_dependents telephone foreign response
101 3 24 2 1 0 0 0 0 0 1469 1 4 4 0 0 1 0 0 3 1 0 41 0 1 0 1 1 1 0 0 good
102 1 36 2 0 0 0 1 0 0 2323 0 3 4 0 1 0 0 0 3 0 0 24 0 1 0 1 2 1 0 0 good
103 3 6 3 0 0 0 1 0 0 932 0 2 3 0 0 0 0 0 1 1 0 24 0 0 1 1 2 1 0 0 good
104 1 9 4 0 0 1 0 0 0 1919 0 3 4 0 1 0 0 0 2 0 0 35 0 1 0 1 2 1 1 0 good
105 3 12 2 0 1 0 0 0 0 2445 4 1 2 0 0 1 0 0 3 0 0 26 0 1 0 1 2 1 1 0 good
106 1 24 4 0 0 0 0 0 0 11938 0 2 2 0 1 0 1 0 2 0 0 39 0 0 1 2 3 2 1 0 bad
107 3 18 1 1 0 0 0 0 0 6458 0 4 2 0 1 0 0 0 3 0 1 39 1 0 1 2 3 2 1 0 bad
108 1 12 2 1 0 0 0 0 0 6078 0 3 2 0 1 0 0 0 1 0 0 32 0 0 1 1 2 1 0 0 good
109 0 24 2 0 0 1 0 0 0 7721 4 1 1 0 0 0 0 0 1 0 0 30 0 0 1 1 2 1 1 1 good
110 1 14 2 0 0 0 0 0 1 1410 2 4 1 0 0 1 0 0 1 1 0 35 0 0 1 1 2 1 1 0 good
111 1 6 3 0 0 0 0 0 1 1449 1 4 1 1 0 0 0 0 1 0 0 31 1 0 1 2 2 2 0 0 good
112 2 15 2 0 0 0 0 1 0 392 0 1 4 0 0 0 0 0 3 0 0 23 0 1 0 1 2 1 1 0 good
113 1 18 2 1 0 0 0 0 0 6260 0 3 3 0 1 0 0 0 2 1 0 28 0 1 0 1 1 1 0 0 good
114 3 36 4 1 0 0 0 0 0 7855 0 2 4 0 0 0 0 0 1 1 0 25 1 0 1 2 2 1 1 0 bad
115 0 12 2 0 0 0 1 0 0 1680 2 4 3 0 0 1 0 0 0 1 0 35 0 0 1 1 2 1 0 0 good
116 3 48 4 0 0 0 1 0 0 3578 4 4 4 0 1 0 0 0 0 1 0 47 0 0 1 1 2 1 1 0 good
117 0 42 2 0 0 0 1 0 0 7174 4 3 4 0 0 0 0 0 2 0 0 30 0 0 1 1 3 1 1 0 bad
118 0 10 4 0 0 1 0 0 0 2132 4 1 2 0 0 0 1 0 2 1 0 27 0 1 0 2 2 1 0 1 good
119 0 33 4 0 0 1 0 0 0 4281 2 2 1 0 0 0 0 0 3 0 0 23 0 0 1 2 2 1 0 0 bad
120 1 12 4 1 0 0 0 0 0 2366 2 3 3 1 0 0 0 0 2 0 0 36 0 0 1 1 3 1 1 0 good
121 0 21 2 0 0 0 1 0 0 1835 0 2 3 0 0 0 0 0 1 1 0 25 0 0 1 2 2 1 1 0 bad
122 3 24 4 0 1 0 0 0 0 3868 0 4 4 0 0 0 0 0 1 0 0 41 0 1 0 2 3 1 1 0 good
123 3 12 2 0 0 1 0 0 0 1768 0 2 3 0 1 0 0 0 1 1 0 24 0 1 0 1 1 1 0 0 good
124 2 10 4 1 0 0 0 0 0 781 0 4 4 0 1 0 0 0 3 0 1 63 0 0 0 2 2 1 1 0 good
125 1 18 2 0 0 1 0 0 0 1924 4 1 4 0 0 0 0 0 2 1 0 27 0 1 0 1 2 1 0 0 bad
126 0 12 4 1 0 0 0 0 0 2121 0 2 4 0 1 0 0 0 1 0 0 30 0 0 1 2 2 1 0 0 good
127 0 12 2 0 0 0 1 0 0 701 0 2 4 0 0 1 0 0 1 1 0 40 0 0 1 1 1 1 0 0 good
128 1 12 2 0 0 0 0 0 0 639 0 2 4 0 1 0 0 0 1 0 0 30 0 0 1 1 2 1 0 0 bad
129 1 12 4 0 1 0 0 0 0 1860 0 0 4 0 1 0 0 0 1 0 0 34 0 0 1 2 3 1 1 0 good
130 0 12 4 1 0 0 0 0 0 3499 0 2 3 0 0 0 1 0 1 1 0 29 0 0 1 2 2 1 0 0 bad
131 1 48 2 1 0 0 0 0 0 8487 4 3 1 0 0 0 0 0 1 0 0 24 0 0 1 1 2 1 0 0 good
132 0 36 3 0 0 0 0 1 0 6887 0 2 4 0 1 0 0 0 2 0 0 29 1 0 1 1 2 1 1 0 bad
133 3 15 2 0 0 1 0 0 0 2708 0 1 2 0 1 0 0 0 2 0 0 27 1 0 1 2 1 1 0 0 good
134 3 18 2 0 0 1 0 0 0 1984 0 2 4 0 1 0 0 0 3 0 1 47 1 0 0 2 2 1 0 0 good
135 3 60 2 0 0 0 1 0 0 10144 1 3 2 0 0 0 0 0 3 1 0 21 0 0 1 1 2 1 1 0 good
136 3 12 4 0 0 0 1 0 0 1240 4 4 4 0 0 0 0 0 1 1 0 38 0 0 1 2 2 1 1 0 good
137 3 27 3 0 1 0 0 0 0 8613 3 2 2 0 1 0 0 0 1 0 0 27 0 0 1 2 2 1 0 0 good
138 1 12 2 0 0 0 1 0 0 766 2 2 4 0 1 0 0 0 2 1 0 66 0 0 1 1 1 1 0 0 bad
139 1 15 4 0 0 0 1 0 0 2728 4 3 4 0 1 0 0 1 1 1 0 35 1 0 1 3 2 1 1 0 good
140 2 12 2 0 0 0 1 0 0 1881 0 2 2 0 0 0 0 0 1 0 0 44 0 1 0 1 1 1 1 0 good
141 2 6 2 1 0 0 0 0 0 709 3 1 2 0 0 1 0 0 1 1 0 27 0 0 1 1 0 1 0 1 good
142 1 36 2 0 0 0 1 0 0 4795 0 1 4 0 0 0 0 0 0 0 1 30 0 0 1 1 3 1 1 0 good
143 0 27 2 0 0 0 1 0 0 3416 0 2 3 0 1 0 0 0 1 0 0 27 0 0 1 1 3 1 0 0 good
144 0 18 2 0 0 1 0 0 0 2462 0 2 2 0 1 0 0 0 1 0 0 22 0 0 1 1 2 1 0 0 bad
145 3 21 4 0 0 1 0 0 0 2288 0 1 4 0 0 0 0 0 3 0 0 23 0 0 1 1 2 1 1 0 good
146 1 48 1 0 0 0 0 0 1 3566 1 3 4 0 1 0 0 0 1 0 0 30 0 0 1 1 2 1 0 0 good
147 0 6 4 1 0 0 0 0 0 860 0 4 1 0 0 0 0 0 3 0 1 39 0 0 1 2 2 1 1 0 good
148 3 12 4 1 0 0 0 0 0 682 1 3 4 0 0 0 0 0 2 0 0 51 0 0 1 2 2 1 1 0 good
149 0 36 4 0 0 1 0 0 0 5371 0 2 3 0 1 0 0 1 1 0 0 28 0 0 1 2 2 1 0 0 good
150 3 18 4 0 0 0 1 0 0 1582 3 4 4 0 1 0 0 0 3 0 0 46 0 0 1 2 2 1 0 0 good
151 3 6 2 0 0 0 1 0 0 1346 1 4 2 0 1 0 0 0 3 0 1 42 1 0 0 1 2 2 1 0 good
152 3 10 2 0 0 0 1 0 0 1924 0 2 1 0 1 0 0 0 3 0 0 38 0 0 1 1 2 1 1 1 good
153 2 36 2 0 0 0 1 0 0 5848 0 2 4 0 1 0 0 0 0 0 0 24 0 0 1 1 2 1 0 0 good
154 1 24 4 0 1 0 0 0 0 7758 3 4 2 0 0 0 0 0 3 0 1 29 0 1 0 1 2 1 0 0 good
155 1 24 3 0 0 0 0 0 1 6967 1 3 4 0 1 0 0 0 3 0 0 36 0 1 0 1 3 1 1 0 good
156 0 12 2 0 0 1 0 0 0 1282 0 2 2 0 0 0 0 0 3 0 0 20 0 1 0 1 2 1 0 0 bad
157 0 9 4 0 0 0 0 0 0 1288 1 4 3 0 1 0 0 1 3 1 0 48 0 0 1 2 2 2 0 1 good
158 0 12 1 0 0 0 0 0 0 339 0 4 4 0 0 1 0 0 0 0 0 45 1 0 1 1 1 1 0 0 good
159 1 24 2 1 0 0 0 0 0 3512 1 3 2 0 1 0 0 0 2 0 0 38 1 0 1 2 2 1 1 0 good
160 3 6 4 0 0 0 1 0 0 1898 4 2 1 0 1 0 0 0 1 1 0 34 0 0 1 2 1 2 0 0 good
161 3 24 4 0 0 0 1 0 0 2872 1 4 3 0 1 0 0 0 3 1 0 36 0 0 1 1 2 2 1 0 good
162 3 18 4 1 0 0 0 0 0 1055 0 1 4 0 0 0 0 0 0 0 0 30 0 0 1 2 2 1 0 0 good
163 3 15 2 0 0 0 0 0 0 1262 2 3 4 0 1 0 0 0 2 0 0 36 0 0 1 2 2 1 1 0 good
164 1 10 2 1 0 0 0 0 0 7308 0 0 2 0 1 0 0 0 3 0 1 70 1 0 0 1 3 1 1 0 good
165 3 36 2 1 0 0 0 0 0 909 2 4 4 0 1 0 0 0 3 0 0 36 0 0 1 1 2 1 0 0 good
166 3 6 2 0 0 1 0 0 0 2978 2 2 1 0 1 0 0 0 1 0 0 32 0 0 1 1 2 1 1 0 good
167 0 18 2 0 0 1 0 0 0 1131 0 0 4 0 0 0 0 0 1 0 0 33 0 0 1 1 2 1 0 0 bad
168 1 11 2 0 0 1 0 0 0 1577 3 1 4 0 0 0 0 0 0 1 0 20 0 0 1 1 2 1 0 0 good
169 3 24 2 0 0 1 0 0 0 3972 0 3 2 0 0 0 0 0 3 0 0 25 0 1 0 1 2 1 1 0 good
170 1 24 4 0 0 0 0 0 1 1935 0 4 4 1 0 0 0 0 3 1 0 31 0 0 1 2 2 1 1 0 bad
171 0 15 0 1 0 0 0 0 0 950 0 4 4 0 1 0 0 0 2 0 0 33 0 1 0 2 2 2 0 0 bad
172 3 12 2 0 0 1 0 0 0 763 0 2 4 0 0 0 0 0 0 1 0 26 0 0 1 1 2 1 1 0 good
173 1 24 3 0 0 1 0 0 0 2064 0 0 3 0 0 0 0 0 1 0 0 34 0 0 1 1 3 1 1 0 bad
174 1 8 2 0 0 0 1 0 0 1414 0 2 4 0 1 0 0 1 1 1 0 33 0 0 1 1 2 1 0 1 good
175 0 21 3 0 0 0 0 1 0 3414 0 1 2 0 1 0 0 0 0 0 0 26 0 0 1 2 2 1 0 0 bad
176 3 30 1 0 1 0 0 0 0 7485 4 0 4 0 0 0 0 0 0 1 0 53 1 0 1 1 3 1 1 0 bad
177 0 12 2 0 0 1 0 0 0 2577 0 2 2 1 0 0 0 0 0 0 0 42 0 0 1 1 2 1 0 0 good
178 0 6 4 0 0 0 1 0 0 338 2 4 4 0 1 0 0 0 3 0 0 52 0 0 1 2 2 1 0 0 good
179 3 12 2 0 0 0 1 0 0 1963 0 3 4 0 1 0 0 0 1 0 0 31 0 1 0 2 3 2 1 0 good
180 0 21 4 1 0 0 0 0 0 571 0 4 4 0 1 0 0 0 3 1 0 65 0 0 1 2 2 1 0 0 good
181 3 36 3 0 0 0 0 0 1 9572 0 1 1 1 0 0 0 0 0 0 0 28 0 0 1 2 2 1 0 0 bad
182 1 36 3 0 0 0 0 0 1 4455 0 2 2 1 0 0 0 0 1 1 0 30 1 0 1 2 3 1 1 0 bad
183 0 21 1 1 0 0 0 0 0 1647 4 2 4 0 1 0 0 0 1 0 0 40 0 0 1 2 1 2 0 0 bad
184 3 24 4 0 0 1 0 0 0 3777 3 2 4 0 1 0 0 0 3 1 0 50 0 0 1 1 2 1 1 0 good
185 1 18 4 1 0 0 0 0 0 884 0 4 4 0 1 0 0 0 3 0 0 36 1 0 1 1 2 2 1 0 bad
186 3 15 4 0 0 0 1 0 0 1360 0 2 4 0 1 0 0 0 1 0 0 31 0 0 1 2 2 1 0 0 good
187 1 9 1 0 1 0 0 0 0 5129 0 4 2 0 0 0 0 0 3 0 1 74 1 0 0 1 3 2 1 0 bad
188 1 16 4 1 0 0 0 0 0 1175 0 0 2 0 1 0 0 0 2 0 0 68 0 0 0 3 0 1 1 0 good
189 0 12 2 0 0 0 1 0 0 674 1 3 4 0 0 1 0 0 0 0 0 20 0 0 1 1 2 1 0 0 bad
190 1 18 0 0 0 1 0 0 0 3244 0 2 1 0 0 0 0 0 3 0 0 33 1 0 1 2 2 1 1 0 good
191 3 24 2 0 0 0 0 0 1 4591 3 2 2 0 1 0 0 0 2 0 0 54 0 0 1 3 3 1 1 0 bad
192 1 48 0 0 0 0 0 0 1 3844 1 3 4 0 1 0 0 0 3 0 1 34 0 0 0 1 1 2 0 0 bad
193 1 27 2 0 0 0 0 0 1 3915 0 2 4 0 1 0 0 0 1 0 0 36 0 0 1 1 2 2 1 0 bad
194 3 6 2 0 0 0 1 0 0 2108 0 3 2 0 0 1 0 0 1 1 0 29 0 1 0 1 2 1 0 0 good
195 1 45 2 0 0 0 1 0 0 3031 1 2 4 0 1 0 0 1 3 0 0 21 0 1 0 1 2 1 0 0 bad
196 1 9 4 0 0 0 0 1 0 1501 0 4 2 0 0 0 0 0 2 0 0 34 0 0 1 2 3 1 1 0 bad
197 3 6 4 0 0 0 1 0 0 1382 0 2 1 0 0 0 0 0 0 0 0 28 0 0 1 2 2 1 1 0 good
198 1 12 2 0 0 1 0 0 0 951 1 1 4 0 0 0 0 0 3 0 0 27 1 1 0 4 2 1 0 0 bad
199 1 24 2 0 1 0 0 0 0 2760 4 4 4 0 1 0 0 0 3 0 1 36 1 0 0 1 2 1 1 0 good
200 1 18 3 0 0 1 0 0 0 4297 0 4 4 1 0 0 0 0 2 0 1 40 0 0 1 1 3 1 1 0 bad
201 3 9 4 0 0 0 0 1 0 936 2 4 4 0 1 0 0 0 1 0 0 52 0 0 1 2 2 1 1 0 good
202 0 12 2 1 0 0 0 0 0 1168 0 2 4 0 0 1 0 0 2 1 0 27 0 0 1 1 1 1 0 0 good
203 3 27 3 0 0 0 0 0 1 5117 0 3 3 0 1 0 0 0 3 0 0 26 0 0 1 2 2 1 0 0 good
204 0 12 2 0 0 0 0 0 0 902 0 3 4 0 0 1 0 0 3 0 0 21 0 1 0 1 2 1 0 0 bad
205 3 12 4 1 0 0 0 0 0 1495 0 4 4 0 1 0 0 0 0 1 0 38 0 0 1 2 1 2 0 0 good
206 0 30 4 0 1 0 0 0 0 10623 0 4 3 0 1 0 0 0 3 0 1 38 0 0 0 3 3 2 1 0 good
207 3 12 4 0 0 1 0 0 0 1935 0 4 4 0 1 0 0 0 3 1 0 43 0 0 1 3 2 1 1 0 good
208 1 12 4 0 0 0 0 0 0 1424 0 3 4 0 1 0 0 0 2 0 0 26 0 0 1 1 2 1 0 0 good
209 0 24 2 0 0 0 0 0 1 6568 0 2 2 0 0 1 0 0 1 0 0 21 1 0 1 1 1 1 0 0 good
210 3 12 2 0 1 0 0 0 0 1413 3 3 3 0 1 0 0 0 1 0 0 55 0 0 1 1 2 1 0 1 good
211 3 9 4 0 0 0 1 0 0 3074 4 2 1 0 1 0 0 0 1 1 0 33 0 0 1 2 2 2 0 0 good
212 3 36 2 0 0 0 1 0 0 3835 4 4 2 0 0 0 0 0 3 1 0 45 0 0 1 1 1 1 1 0 good
213 0 27 0 0 0 0 0 0 1 5293 0 0 2 0 1 0 0 0 3 0 0 50 1 0 1 2 2 1 1 0 bad
214 2 30 3 0 0 0 0 0 1 1908 0 4 4 0 1 0 0 0 3 1 0 66 0 0 1 1 3 1 1 0 bad
215 3 36 4 0 0 0 1 0 0 3342 4 4 4 0 1 0 0 0 1 0 0 51 0 0 1 1 2 1 1 0 good
216 1 6 4 0 0 0 0 0 0 932 4 3 1 0 0 0 0 0 2 0 0 39 0 0 1 2 1 1 0 0 good
217 0 18 0 0 0 0 0 0 1 3104 0 3 3 0 1 0 0 0 0 0 0 31 1 0 1 1 2 1 1 0 good
218 2 36 2 0 0 0 1 0 0 3913 0 2 2 0 1 0 0 0 1 1 0 23 0 0 1 1 2 1 1 0 good
219 0 24 2 0 0 1 0 0 0 3021 0 2 2 1 0 0 0 0 1 1 0 24 0 1 0 1 1 1 0 0 good
220 3 10 2 1 0 0 0 0 0 1364 0 2 2 0 0 0 0 0 3 0 0 64 0 0 1 1 2 1 1 0 good
221 1 12 2 0 0 0 1 0 0 625 0 1 4 0 0 1 0 1 0 1 0 26 1 0 1 1 1 1 0 0 good
222 0 12 2 0 0 0 0 1 0 1200 4 2 4 0 0 0 0 0 3 0 0 23 1 1 0 1 2 1 1 0 good
223 3 12 2 0 0 0 1 0 0 707 0 2 4 0 1 0 0 0 1 1 0 30 1 0 1 2 2 1 0 0 good
224 3 24 3 0 0 0 0 0 1 2978 4 2 4 0 1 0 0 0 3 1 0 32 0 0 1 2 2 2 1 0 good
225 3 15 2 0 1 0 0 0 0 4657 0 2 3 0 1 0 0 0 1 0 0 30 0 0 1 1 2 1 1 0 good
226 3 36 0 0 0 0 0 0 0 2613 0 2 4 0 1 0 0 0 1 0 0 27 0 0 1 2 2 1 0 0 good
227 1 48 2 0 0 0 1 0 0 10961 3 3 1 0 1 0 1 0 1 0 1 27 1 0 1 2 2 1 1 0 bad
228 0 12 2 0 0 1 0 0 0 7865 0 4 4 0 1 0 0 0 3 0 1 53 0 0 0 1 3 1 1 0 bad
229 3 9 2 0 0 0 1 0 0 1478 0 3 4 0 1 0 0 0 1 0 0 22 0 0 1 1 2 1 0 0 bad
230 0 24 2 0 0 1 0 0 0 3149 0 1 4 0 1 0 0 0 0 0 1 22 1 0 0 1 2 1 0 0 good
231 2 36 2 0 0 0 1 0 0 4210 0 2 4 0 1 0 0 0 1 0 0 26 0 0 1 1 2 1 0 0 bad
232 3 9 2 1 0 0 0 0 0 2507 2 4 2 0 1 0 0 0 3 0 1 51 0 0 0 1 1 1 0 0 good
233 3 12 2 0 0 0 1 0 0 2141 1 3 3 0 1 0 0 0 0 0 1 35 0 0 1 1 2 1 0 0 good
234 1 18 2 0 0 0 1 0 0 866 0 2 4 0 0 1 0 0 1 1 0 25 0 0 1 1 1 1 0 0 good
235 3 4 4 0 0 0 1 0 0 1544 0 3 2 0 1 0 0 0 0 1 0 42 0 0 1 3 1 2 0 0 good
236 0 24 2 0 0 0 1 0 0 1823 0 0 4 0 1 0 0 0 1 0 0 30 1 0 1 1 3 2 0 0 bad
237 1 6 2 1 0 0 0 0 0 14555 4 0 1 0 1 0 0 0 1 0 0 23 0 0 1 1 0 1 1 0 bad
238 1 21 2 0 0 0 0 0 1 2767 1 4 4 1 0 0 0 0 1 0 0 61 1 1 0 2 1 1 0 0 bad
239 3 12 4 0 0 0 1 0 0 1291 0 2 4 0 0 0 0 0 1 0 0 35 0 0 1 2 2 1 0 0 good
240 0 30 2 0 0 0 1 0 0 2522 0 4 1 0 1 0 0 1 2 0 0 39 0 0 1 1 2 2 0 0 good
241 0 24 2 1 0 0 0 0 0 915 4 4 4 0 0 0 0 0 1 0 0 29 1 0 1 1 2 1 0 0 bad
242 3 6 2 0 0 0 1 0 0 1595 0 3 3 0 1 0 0 0 1 0 0 51 0 0 1 1 2 2 0 0 good
243 0 48 0 0 1 0 0 0 0 4605 0 4 3 0 1 0 0 0 3 0 1 24 0 0 0 2 2 2 0 0 bad
244 3 12 4 0 0 0 0 0 1 1185 0 2 3 0 0 0 0 0 1 1 0 27 0 0 1 2 2 1 0 0 good
245 3 12 1 0 0 0 0 0 0 3447 2 2 4 0 0 0 0 0 2 1 0 35 0 0 1 1 1 2 0 0 good
246 3 24 2 0 0 0 0 0 1 1258 0 3 4 0 1 0 0 0 0 1 0 25 0 0 1 1 2 1 1 0 good
247 3 12 4 0 0 0 1 0 0 717 0 4 4 0 1 0 0 0 3 1 0 52 0 0 1 3 2 1 0 0 good
248 3 6 0 1 0 0 0 0 0 1204 1 2 4 0 1 0 0 0 0 0 1 35 1 1 0 1 2 1 0 1 good
249 2 24 2 0 0 1 0 0 0 1925 0 2 2 0 1 0 0 0 1 1 0 26 0 0 1 1 2 1 0 0 good
250 3 18 2 0 0 0 1 0 0 433 0 0 3 0 0 0 1 0 3 1 0 22 0 1 0 1 2 1 0 0 bad
251 0 6 4 1 0 0 0 0 0 666 3 3 3 0 0 0 0 0 3 1 0 39 0 0 1 2 1 1 1 0 good
252 2 12 2 0 0 1 0 0 0 2251 0 2 1 0 0 0 0 0 1 0 0 46 0 0 1 1 1 1 0 0 good
253 1 30 2 1 0 0 0 0 0 2150 0 2 4 0 0 0 0 1 1 0 1 24 1 0 1 1 2 1 0 0 bad
254 3 24 3 0 0 1 0 0 0 4151 1 2 2 0 1 0 0 0 2 0 0 35 0 0 1 2 2 1 0 0 good
255 1 9 2 0 0 1 0 0 0 2030 4 3 2 0 1 0 0 0 0 0 0 24 0 0 1 1 2 1 1 0 good
256 1 60 3 0 0 0 1 0 0 7418 4 2 1 0 1 0 0 0 0 1 0 27 0 0 1 1 1 1 0 0 good
257 3 24 4 0 0 0 1 0 0 2684 0 2 4 0 1 0 0 0 1 1 0 35 0 0 1 2 1 1 0 0 good
258 0 12 1 0 0 0 1 0 0 2149 0 2 4 1 0 0 0 0 0 0 1 29 0 0 0 1 2 1 0 0 bad
259 3 15 2 0 1 0 0 0 0 3812 1 1 1 0 0 0 0 0 3 0 0 23 0 0 1 1 2 1 1 0 good
260 3 11 4 0 0 0 1 0 0 1154 1 0 4 0 0 0 0 0 3 1 0 57 0 0 1 3 1 1 0 0 good
261 0 12 2 0 0 1 0 0 0 1657 0 2 2 0 1 0 0 0 1 1 0 27 0 0 1 1 2 1 0 0 good
262 0 24 2 0 0 0 1 0 0 1603 0 4 4 0 0 0 0 0 3 0 0 55 0 0 1 1 2 1 0 0 good
263 0 18 4 1 0 0 0 0 0 5302 0 4 2 0 1 0 0 0 3 0 1 36 0 0 0 3 3 1 1 0 good
264 3 12 4 0 0 0 0 1 0 2748 0 4 2 0 0 0 0 0 3 0 1 57 1 0 0 3 1 1 0 0 good
265 3 10 4 1 0 0 0 0 0 1231 0 4 3 0 1 0 0 0 3 1 0 32 0 0 1 2 1 2 0 1 good
266 1 15 2 0 0 0 1 0 0 802 0 4 4 0 1 0 0 0 2 0 0 37 0 0 1 1 2 2 0 0 bad
267 3 36 4 0 0 0 0 0 1 6304 4 4 4 0 1 0 0 0 3 1 0 36 0 0 1 2 2 1 0 0 good
268 3 24 2 0 0 0 1 0 0 1533 0 1 4 0 0 0 0 0 2 0 0 38 1 0 1 1 2 1 1 0 good
269 0 14 2 1 0 0 0 0 0 8978 0 4 1 1 0 0 0 0 3 0 0 45 0 0 1 1 3 1 1 1 bad
270 3 24 2 0 0 0 1 0 0 999 4 4 4 0 1 0 0 0 1 0 0 25 0 0 1 2 2 1 0 0 good
271 3 18 2 1 0 0 0 0 0 2662 4 3 4 0 1 0 0 0 2 0 0 32 0 0 1 1 2 1 0 1 good
272 3 12 4 0 0 1 0 0 0 1402 2 3 3 0 0 0 0 0 3 0 0 37 0 1 0 1 2 1 1 0 good
273 1 48 1 1 0 0 0 0 0 12169 4 0 4 0 1 0 1 0 3 0 1 36 0 0 0 1 3 1 1 0 good
274 1 48 2 0 0 0 1 0 0 3060 0 3 4 0 1 0 0 0 3 1 0 28 0 0 1 2 2 1 0 0 bad
275 0 30 2 0 0 0 0 0 0 11998 0 1 1 1 0 0 0 0 0 0 1 34 0 0 1 1 1 1 1 0 bad
276 3 9 2 0 0 0 1 0 0 2697 0 2 1 0 1 0 0 0 1 1 0 32 0 0 1 1 2 2 0 0 good
277 3 18 4 0 0 0 1 0 0 2404 0 2 2 0 0 0 0 0 1 0 0 26 0 0 1 2 2 1 0 0 good
278 0 12 2 0 0 1 0 0 0 1262 4 4 2 1 0 0 0 0 3 0 0 49 0 0 1 1 1 1 1 0 good
279 3 6 2 0 0 1 0 0 0 4611 0 1 1 0 0 0 0 0 3 0 0 32 0 0 1 1 2 1 0 0 bad
280 3 24 2 0 0 0 1 0 0 1901 1 2 4 0 1 0 0 0 3 0 0 29 0 1 0 1 3 1 1 0 good
281 3 15 4 0 1 0 0 0 0 3368 3 4 3 0 1 0 0 0 3 0 1 23 0 1 0 2 2 1 1 0 good
282 3 12 2 0 0 1 0 0 0 1574 0 2 4 0 1 0 0 0 1 1 0 50 0 0 1 1 2 1 0 0 good
283 2 18 1 0 0 0 1 0 0 1445 4 3 4 0 1 0 0 0 3 0 0 49 1 0 1 1 1 1 0 0 good
284 3 15 4 0 0 1 0 0 0 1520 4 4 4 0 1 0 0 0 3 0 0 63 0 0 1 1 2 1 0 0 good
285 1 24 4 1 0 0 0 0 0 3878 1 1 4 1 0 0 0 0 1 0 0 37 0 0 1 1 2 1 1 0 good
286 0 47 2 1 0 0 0 0 0 10722 0 1 1 0 0 0 0 0 0 1 0 35 0 0 1 1 1 1 1 0 good
287 0 48 2 0 1 0 0 0 0 4788 0 3 4 0 1 0 0 0 2 0 0 26 0 0 1 1 2 2 0 0 good
288 1 48 3 0 0 0 0 0 0 7582 1 0 2 0 1 0 0 0 3 0 1 31 0 0 0 1 3 1 1 0 good
289 1 12 2 0 0 0 1 0 0 1092 0 2 4 0 0 0 0 1 3 1 0 49 0 0 1 2 2 1 1 0 good
290 0 24 3 0 0 0 1 0 0 1024 0 1 4 0 0 1 0 0 3 1 0 48 1 0 1 1 2 1 0 0 bad
291 3 12 2 0 0 0 0 0 1 1076 0 2 2 0 0 1 0 0 1 1 0 26 0 0 1 1 2 1 1 1 good
292 1 36 2 0 1 0 0 0 0 9398 0 1 1 0 0 1 0 0 3 0 0 28 0 1 0 1 3 1 1 0 bad
293 0 24 4 0 1 0 0 0 0 6419 0 4 2 0 0 0 0 0 3 0 1 44 0 0 0 2 3 2 1 0 good
294 2 42 4 0 1 0 0 0 0 4796 0 4 4 0 1 0 0 0 3 0 1 56 0 0 0 1 2 1 0 0 good
295 3 48 4 0 0 0 0 0 1 7629 4 4 4 1 0 0 0 0 1 0 0 46 1 0 1 2 3 2 0 0 good
296 1 48 2 0 0 1 0 0 0 9960 0 1 1 0 0 0 0 0 1 0 0 26 0 0 1 1 2 1 1 0 bad
297 3 12 2 0 1 0 0 0 0 4675 4 1 1 0 0 0 0 0 3 0 0 20 0 1 0 1 2 1 0 0 good
298 3 10 2 1 0 0 0 0 0 1287 4 4 4 0 1 0 1 0 1 0 0 45 0 0 1 1 1 1 0 1 good
299 3 18 2 0 0 1 0 0 0 2515 0 2 3 0 1 0 0 0 3 1 0 43 0 0 1 1 2 1 1 0 good
300 1 21 4 0 0 1 0 0 0 2745 3 3 3 0 1 0 0 0 1 0 0 32 0 0 1 2 2 1 1 0 good
301 3 6 2 1 0 0 0 0 0 672 0 0 1 0 0 0 0 0 3 1 0 54 0 0 1 1 0 1 1 0 good
302 1 36 0 0 0 0 1 0 0 3804 0 2 4 0 0 0 0 0 0 0 0 42 0 0 1 1 2 1 1 0 bad
303 2 24 4 1 0 0 0 0 0 1344 4 3 4 0 1 0 0 0 1 1 0 37 1 0 1 2 1 2 0 0 bad
304 0 10 4 1 0 0 0 0 0 1038 0 3 4 0 1 0 1 0 2 0 0 49 0 0 1 2 2 1 1 0 good
305 3 48 4 1 0 0 0 0 0 10127 2 2 2 0 1 0 0 0 1 0 1 44 1 0 0 1 2 1 0 0 bad
306 3 6 2 0 0 1 0 0 0 1543 3 2 4 1 0 0 0 0 1 1 0 33 0 0 1 1 2 1 0 0 good
307 3 30 2 0 1 0 0 0 0 4811 4 3 2 0 0 0 0 0 3 0 0 24 1 1 0 1 1 1 0 0 good
308 0 12 2 0 0 0 1 0 0 727 1 1 4 0 0 1 0 0 2 0 1 33 0 0 1 1 1 1 1 0 bad
309 1 8 2 0 0 1 0 0 0 1237 0 2 3 0 0 0 0 0 3 1 0 24 0 0 1 1 2 1 0 0 bad
310 1 9 2 1 0 0 0 0 0 276 0 2 4 0 0 1 0 0 3 1 0 22 0 1 0 1 1 1 0 0 good
311 1 48 2 0 0 0 0 0 0 5381 4 0 3 0 1 0 0 0 3 0 1 40 1 0 0 1 0 1 1 0 good
312 3 24 2 0 0 1 0 0 0 5511 1 2 4 0 1 0 0 0 0 0 0 25 1 0 1 1 2 1 0 0 good
313 2 24 2 0 0 1 0 0 0 3749 0 1 2 0 0 0 0 0 3 0 0 26 0 0 1 1 2 1 0 0 good
314 1 12 2 1 0 0 0 0 0 685 0 3 2 0 0 1 0 0 2 0 0 25 1 0 1 1 1 1 0 0 bad
315 2 4 2 1 0 0 0 0 0 1494 4 1 1 0 1 0 0 0 1 1 0 29 0 0 1 1 1 2 0 1 good
316 0 36 1 0 0 1 0 0 0 2746 0 4 4 0 1 0 0 0 3 0 0 31 1 0 1 1 2 1 0 0 bad
317 0 12 2 0 0 1 0 0 0 708 0 2 2 0 1 0 0 1 2 0 0 38 0 0 1 1 1 2 0 0 good
318 1 24 2 0 0 1 0 0 0 4351 4 2 1 0 0 0 0 0 3 0 0 48 0 0 1 1 1 1 1 0 good
319 3 12 4 0 0 0 0 1 0 701 0 2 4 0 1 0 0 0 1 0 0 32 0 0 1 2 2 1 0 0 good
320 0 15 3 0 0 1 0 0 0 3643 0 4 1 0 0 0 0 0 3 0 0 27 0 0 1 2 1 1 0 0 good
321 1 30 4 1 0 0 0 0 0 4249 0 0 4 0 0 1 0 0 1 0 0 28 0 0 1 2 3 1 0 0 bad
322 0 24 2 0 0 0 1 0 0 1938 0 1 4 1 0 0 0 0 2 0 0 32 0 0 1 1 2 1 0 0 bad
323 0 24 2 0 1 0 0 0 0 2910 0 3 2 0 1 0 0 0 0 0 1 34 0 0 0 1 3 1 1 0 good
324 0 18 2 0 0 1 0 0 0 2659 3 2 4 0 1 0 0 0 1 0 0 28 0 0 1 1 2 1 0 0 good
325 3 18 4 1 0 0 0 0 0 1028 0 2 4 0 0 0 0 0 2 1 0 36 0 0 1 2 2 1 0 0 good
326 0 8 4 1 0 0 0 0 0 3398 0 3 1 0 1 0 0 0 3 1 0 39 0 0 1 2 1 1 0 1 good
327 3 12 4 0 0 1 0 0 0 5801 4 4 2 0 1 0 0 0 3 0 0 49 0 1 0 1 2 1 1 0 good
328 3 24 2 1 0 0 0 0 0 1525 3 3 4 0 0 0 0 0 2 0 0 34 0 0 1 1 2 2 1 0 good
329 2 36 2 0 0 0 1 0 0 4473 0 4 4 0 1 0 0 0 1 0 0 31 0 0 1 1 2 1 0 0 good
330 1 6 2 0 0 0 1 0 0 1068 0 4 4 0 1 0 0 0 3 0 0 28 0 0 1 1 2 2 0 0 good
331 0 24 4 0 1 0 0 0 0 6615 0 0 2 0 1 0 0 0 3 0 1 75 0 0 0 2 3 1 1 0 good
332 3 18 4 0 0 0 0 1 0 1864 1 2 4 0 0 0 0 0 1 1 0 30 0 0 1 2 2 1 0 0 bad
333 1 60 2 1 0 0 0 0 0 7408 1 1 4 0 0 0 0 0 1 0 0 24 0 0 1 1 3 1 0 0 bad
334 3 48 4 0 1 0 0 0 0 11590 1 2 2 0 0 0 0 0 3 0 0 24 1 1 0 2 1 1 0 0 bad
335 0 24 0 0 0 1 0 0 0 4110 0 4 3 0 1 0 0 0 3 0 1 23 1 1 0 2 2 2 0 0 bad
336 0 6 4 0 0 1 0 0 0 3384 0 2 1 1 0 0 0 0 3 1 0 44 0 1 0 1 3 1 1 0 bad
337 1 13 2 0 0 0 1 0 0 2101 0 1 2 0 0 0 0 1 3 0 0 23 0 0 1 1 1 1 0 0 good
338 0 15 2 0 0 0 0 0 0 1275 4 2 4 0 0 0 0 0 1 0 0 24 0 1 0 1 2 1 0 0 bad
339 0 24 2 0 0 1 0 0 0 4169 0 2 4 0 1 0 0 0 3 0 0 28 0 0 1 1 2 1 0 0 good
340 1 10 2 0 0 1 0 0 0 1521 0 2 4 1 0 0 0 0 1 0 0 31 0 0 1 1 1 1 0 0 good
341 1 24 4 0 0 0 0 1 0 5743 0 1 2 0 0 0 0 0 3 0 1 24 0 0 0 2 2 1 1 0 good
342 0 21 2 0 0 1 0 0 0 3599 0 3 1 0 0 0 0 0 3 0 0 26 0 1 0 1 1 1 0 0 good
343 1 18 2 0 0 0 1 0 0 3213 2 1 1 0 0 1 0 0 2 1 0 25 0 1 0 1 2 1 0 0 good
344 1 18 2 0 0 0 0 0 1 4439 0 4 1 0 1 0 1 0 0 1 0 33 1 0 1 1 3 1 1 0 good
345 2 10 2 1 0 0 0 0 0 3949 0 1 1 0 1 0 0 1 0 0 0 37 0 0 1 1 1 2 0 0 good
346 3 15 4 0 0 0 1 0 0 1459 0 2 4 0 0 0 0 0 1 0 0 43 0 0 1 1 1 1 0 0 good
347 1 13 4 0 0 0 1 0 0 882 0 1 4 0 1 0 0 1 3 1 0 23 0 0 1 2 2 1 0 0 good
348 1 24 2 0 0 0 1 0 0 3758 2 0 1 0 0 0 0 0 3 0 1 23 0 1 0 1 0 1 0 0 good
349 3 6 3 0 0 0 0 0 1 1743 1 2 1 0 1 0 0 0 1 1 0 34 0 0 1 2 1 1 0 0 good
350 1 9 4 0 0 0 0 1 0 1136 3 4 4 0 1 0 0 0 2 0 1 32 0 0 0 2 2 2 0 0 bad
351 3 9 2 0 0 0 0 0 0 1236 0 1 1 0 0 0 0 0 3 1 0 23 0 1 0 1 2 1 1 0 good
352 1 9 2 0 0 1 0 0 0 959 0 2 1 0 0 0 0 0 1 0 0 29 0 0 1 1 2 1 0 1 bad
353 3 18 4 0 1 0 0 0 0 3229 4 0 2 0 1 0 0 0 3 0 1 38 0 0 1 1 3 1 1 0 good
354 0 12 0 0 0 0 1 0 0 6199 0 2 4 0 1 0 0 0 1 0 0 28 0 1 0 2 2 1 1 0 bad
355 3 10 2 0 0 0 0 1 0 727 2 4 4 0 1 0 0 0 3 0 1 46 0 0 0 1 2 1 1 0 good
356 1 24 2 1 0 0 0 0 0 1246 0 1 4 0 1 0 0 0 1 1 0 23 1 0 1 1 1 1 0 0 bad
357 3 12 4 0 0 0 1 0 0 2331 4 4 1 0 1 0 1 0 3 1 0 49 0 0 1 1 2 1 1 0 good
358 3 36 3 0 0 0 1 0 0 4463 0 2 4 0 1 0 0 0 1 0 0 26 0 0 1 2 3 1 1 0 bad
359 3 12 2 0 0 0 1 0 0 776 0 2 4 0 0 1 0 0 1 1 0 28 0 0 1 1 2 1 0 0 good
360 0 30 2 0 0 1 0 0 0 2406 0 3 4 0 0 0 0 0 3 1 0 23 0 1 0 1 2 1 0 0 bad
361 1 18 2 0 0 0 0 1 0 1239 4 2 4 0 1 0 0 0 3 0 1 61 0 0 0 1 2 1 0 0 good
362 2 12 2 0 0 0 1 0 0 3399 4 4 2 0 1 0 0 0 2 0 0 37 0 0 1 1 3 1 0 0 good
363 2 12 3 1 0 0 0 0 0 2247 0 2 2 0 0 0 0 0 1 0 0 36 1 0 1 2 2 1 1 0 good
364 3 6 2 0 0 1 0 0 0 1766 0 2 1 0 0 1 0 0 1 0 0 21 0 1 0 1 2 1 0 0 good
365 0 18 2 0 0 1 0 0 0 2473 0 0 4 0 1 0 0 0 0 0 0 25 0 0 1 1 0 1 0 0 bad
366 3 12 2 0 0 0 0 0 1 1542 0 3 2 0 1 0 0 0 3 0 0 36 0 0 1 1 2 1 1 0 good
367 3 18 4 0 1 0 0 0 0 3850 0 3 3 0 1 0 0 0 0 0 0 27 0 0 1 2 2 1 0 0 good
368 0 18 2 0 0 1 0 0 0 3650 0 1 1 0 0 0 0 0 3 0 0 22 0 1 0 1 2 1 0 0 good
369 0 36 2 0 0 1 0 0 0 3446 0 4 4 0 1 0 0 0 1 0 0 42 0 0 1 1 2 2 0 0 bad
370 1 18 2 0 0 1 0 0 0 3001 0 3 2 0 0 0 0 0 3 1 0 40 0 1 0 1 2 1 0 0 good
371 3 36 2 1 0 0 0 0 0 3079 4 2 4 0 1 0 0 0 3 1 0 36 0 0 1 1 2 1 0 0 good
372 3 18 4 0 0 0 1 0 0 6070 0 4 3 0 1 0 0 0 3 0 0 33 0 0 1 2 2 1 1 0 good
373 3 10 4 0 0 1 0 0 0 2146 0 1 1 0 0 0 0 0 2 1 0 23 0 1 0 2 2 1 0 0 good
374 3 60 4 1 0 0 0 0 0 13756 4 4 2 0 1 0 0 0 3 0 1 63 1 0 0 1 3 1 1 0 good
375 1 60 1 0 0 0 0 0 0 14782 1 4 3 0 0 0 0 0 3 0 1 60 1 0 0 2 3 1 1 0 bad
376 0 48 1 0 0 0 0 0 1 7685 0 3 2 0 0 0 0 1 3 0 0 37 0 1 0 1 2 1 0 0 bad
377 3 18 3 0 0 0 1 0 0 2320 0 0 2 0 0 1 0 0 2 1 0 34 0 0 1 2 2 1 0 0 good
378 3 7 3 0 0 0 1 0 0 846 4 4 3 0 1 0 0 0 3 0 1 36 0 0 0 1 2 1 0 0 good
379 1 36 2 1 0 0 0 0 0 14318 0 4 4 0 1 0 0 0 1 0 1 57 0 0 0 1 3 1 1 0 bad
380 3 6 4 1 0 0 0 0 0 362 1 2 4 0 0 0 0 0 3 0 0 52 0 0 1 2 1 1 0 0 good
381 0 20 2 0 0 1 0 0 0 2212 4 3 4 0 1 0 0 0 3 0 0 39 0 0 1 1 2 1 1 0 good
382 1 18 2 0 1 0 0 0 0 12976 0 0 3 0 0 0 0 0 3 0 1 38 0 0 0 1 3 1 1 0 bad
383 3 22 2 1 0 0 0 0 0 1283 4 3 4 0 0 0 0 0 3 0 0 25 0 1 0 1 2 1 0 0 good
384 2 12 2 1 0 0 0 0 0 1330 0 1 4 0 1 0 0 0 0 1 0 26 0 0 1 1 2 1 0 0 good
385 3 30 3 0 0 0 0 0 1 4272 1 2 2 0 1 0 0 0 1 0 0 26 0 0 1 2 1 1 0 0 good
386 3 18 4 0 0 0 1 0 0 2238 0 2 2 0 0 0 0 0 0 0 0 25 0 0 1 2 2 1 0 0 good
387 3 18 2 0 0 0 1 0 0 1126 4 1 4 0 0 0 0 0 1 1 0 21 0 1 0 1 2 1 1 0 good
388 1 18 4 0 0 1 0 0 0 7374 0 0 4 0 1 0 0 0 3 0 0 40 1 0 1 2 3 1 1 0 good
389 1 15 4 0 0 0 0 0 1 2326 2 2 2 0 1 0 0 0 3 0 0 27 1 0 1 1 2 1 0 0 good
390 3 9 2 0 0 0 0 0 1 1449 0 3 3 0 0 0 0 0 1 0 0 27 0 0 1 2 2 1 0 0 good
391 3 18 2 1 0 0 0 0 0 1820 0 2 2 0 0 1 0 0 1 0 0 30 0 0 1 1 3 1 1 0 good
392 1 12 2 0 0 1 0 0 0 983 3 1 1 0 0 0 0 0 3 1 0 19 0 1 0 1 1 1 0 0 good
393 0 36 2 1 0 0 0 0 0 3249 0 3 2 0 1 0 0 0 3 0 1 39 1 0 0 1 3 2 1 0 good
394 0 6 4 0 0 0 1 0 0 1957 0 3 1 0 0 0 0 0 3 0 0 31 0 0 1 1 2 1 0 0 good
395 3 9 4 0 0 1 0 0 0 2406 0 0 2 0 1 0 0 0 2 0 0 31 0 0 1 1 3 1 0 0 good
396 1 39 3 0 0 0 0 1 0 11760 1 3 2 0 1 0 0 0 2 0 1 32 0 1 0 1 2 1 1 0 good
397 0 12 2 0 0 1 0 0 0 2578 0 0 3 0 0 0 0 0 3 0 1 55 0 0 0 1 3 1 0 0 good
398 0 36 4 0 0 1 0 0 0 2348 0 2 3 0 0 1 0 0 1 0 0 46 0 0 1 2 2 1 1 0 good
399 1 12 2 1 0 0 0 0 0 1223 0 4 1 1 0 0 0 0 0 1 0 46 0 1 0 2 2 1 0 0 bad
400 3 24 4 0 0 0 1 0 0 1516 3 2 4 0 0 0 0 0 0 1 0 43 0 0 1 2 1 1 0 0 good
401 3 18 2 0 0 0 1 0 0 1473 0 1 3 0 0 1 0 0 3 1 0 39 0 0 1 1 2 1 1 0 good
402 1 18 4 0 0 0 0 0 1 1887 4 2 4 0 0 1 0 0 3 1 0 28 1 0 1 2 2 1 0 0 good
403 3 24 3 0 0 0 0 0 1 8648 0 1 2 0 1 0 0 0 1 0 0 27 1 0 1 2 2 1 1 0 bad
404 3 14 3 1 0 0 0 0 0 802 0 2 4 0 1 0 0 0 1 0 0 27 0 0 1 2 1 1 0 0 good
405 1 18 3 1 0 0 0 0 0 2899 4 4 4 0 1 0 0 0 3 0 0 43 0 0 1 1 2 2 0 0 good
406 1 24 2 0 0 0 1 0 0 2039 0 1 1 0 0 1 0 0 0 0 0 22 0 0 1 1 2 1 1 0 bad
407 3 24 4 0 1 0 0 0 0 2197 4 3 4 0 1 0 0 0 3 0 0 43 0 0 1 2 2 2 1 0 good
408 0 15 2 0 0 0 1 0 0 1053 0 1 4 0 0 1 0 0 1 1 0 27 0 0 1 1 2 1 0 1 good
409 3 24 2 0 0 0 1 0 0 3235 2 4 3 1 0 0 0 0 1 0 0 26 0 0 1 1 3 1 1 0 good
410 2 12 4 1 0 0 0 0 0 939 2 3 4 0 0 1 0 0 1 1 0 28 0 0 1 3 2 1 1 0 bad
411 1 24 2 0 0 0 1 0 0 1967 0 4 4 0 0 0 0 0 3 0 0 20 0 0 1 1 2 1 1 0 good
412 3 33 4 0 1 0 0 0 0 7253 0 3 3 0 1 0 0 0 1 0 0 35 0 0 1 2 3 1 1 0 good
413 3 12 4 0 0 0 0 0 1 2292 0 0 4 0 1 0 0 0 1 0 0 42 1 0 1 2 3 1 1 0 bad
414 3 10 2 1 0 0 0 0 0 1597 2 2 3 0 1 0 0 0 1 0 1 40 0 1 0 1 1 2 0 1 good
415 0 24 2 1 0 0 0 0 0 1381 4 2 4 0 0 0 0 0 1 0 0 35 0 0 1 1 2 1 0 0 bad
416 3 36 4 0 1 0 0 0 0 5842 0 4 2 0 1 0 0 0 1 0 0 35 0 0 1 2 2 2 1 0 good
417 0 12 2 1 0 0 0 0 0 2579 0 1 4 0 1 0 0 0 0 1 0 33 0 0 1 1 1 2 0 0 bad
418 0 18 3 0 0 0 0 1 0 8471 4 2 1 0 0 0 0 0 1 0 0 23 0 1 0 2 2 1 1 0 good
419 3 21 2 1 0 0 0 0 0 2782 2 3 1 0 0 0 0 0 1 0 0 31 1 0 1 1 3 1 0 0 good
420 1 18 2 1 0 0 0 0 0 1042 4 2 4 0 0 0 0 0 1 0 0 33 0 0 1 1 2 1 0 0 bad
421 3 15 2 1 0 0 0 0 0 3186 3 3 2 0 0 0 0 0 2 0 0 20 0 1 0 1 2 1 0 0 good
422 1 12 2 0 1 0 0 0 0 2028 4 2 4 0 1 0 0 0 1 0 0 30 0 0 1 1 2 1 0 0 good
423 1 12 4 1 0 0 0 0 0 958 0 3 2 0 1 0 0 0 2 1 0 47 0 0 1 2 1 2 0 0 good
424 3 21 3 0 0 1 0 0 0 1591 1 3 4 0 1 0 0 0 2 1 0 34 0 0 1 2 3 1 0 0 good
425 1 12 2 0 0 1 0 0 0 2762 4 4 1 0 0 0 0 0 1 0 0 25 1 0 1 1 2 1 1 0 bad
426 1 18 2 0 1 0 0 0 0 2779 0 2 1 0 0 1 0 0 2 0 0 21 0 1 0 1 2 1 1 0 good
427 3 28 4 0 0 0 1 0 0 2743 0 4 4 0 1 0 0 0 1 0 0 29 0 0 1 2 2 1 0 0 good
428 3 18 4 0 0 0 1 0 0 1149 3 2 4 0 1 0 0 0 2 1 0 46 0 0 1 2 2 1 0 0 good
429 3 9 2 0 0 1 0 0 0 1313 0 4 1 0 1 0 0 0 3 0 0 20 0 0 1 1 2 1 0 0 good
430 0 18 4 0 0 0 0 0 0 1190 0 0 2 0 0 0 0 0 3 0 1 55 0 0 0 3 0 2 0 0 bad
431 3 5 2 0 0 0 0 0 1 3448 0 3 1 0 1 0 0 0 3 1 0 74 0 0 1 1 1 1 0 0 good
432 1 24 2 0 0 0 0 0 0 11328 0 2 2 0 1 0 1 0 2 0 0 29 1 0 1 2 3 1 1 0 bad
433 0 6 4 0 0 1 0 0 0 1872 0 0 4 0 1 0 0 0 3 0 1 36 0 0 0 3 3 1 1 0 good
434 3 24 4 0 0 0 0 0 0 2058 0 2 4 1 0 0 0 0 1 1 0 33 0 0 1 2 2 1 1 0 good
435 0 9 2 0 0 1 0 0 0 2136 0 2 3 0 1 0 0 0 1 1 0 25 0 0 1 1 2 1 0 0 good
436 1 12 2 0 0 0 1 0 0 1484 4 2 2 0 0 1 0 0 0 1 0 25 0 0 1 1 2 1 1 0 bad
437 3 6 2 0 0 0 0 0 0 660 2 3 2 0 0 1 0 0 3 1 0 23 0 1 0 1 1 1 0 0 good
438 3 24 4 1 0 0 0 0 0 1287 3 4 4 0 0 0 0 0 3 1 0 37 0 0 1 2 2 1 1 0 good
439 0 42 4 0 0 0 0 0 0 3394 0 0 4 0 1 0 1 0 3 0 0 65 0 0 1 2 0 1 0 0 good
440 2 12 1 0 0 0 0 0 1 609 0 1 4 0 0 0 0 0 0 1 0 26 0 0 1 1 0 1 0 0 bad
441 3 12 2 1 0 0 0 0 0 1884 0 4 4 0 1 0 0 0 3 0 0 39 0 0 1 1 3 1 1 0 good
442 0 12 2 0 0 1 0 0 0 1620 0 2 2 0 0 0 1 0 2 0 0 30 0 0 1 1 2 1 0 0 good
443 1 20 3 0 0 0 0 0 0 2629 0 2 2 0 1 0 0 0 2 0 0 29 1 0 1 2 2 1 1 0 good
444 3 12 2 0 0 0 0 1 0 719 0 4 4 0 1 0 0 0 3 0 0 41 1 0 1 1 1 2 0 0 bad
445 1 48 4 0 0 1 0 0 0 5096 0 2 2 0 0 0 0 0 2 0 0 30 0 0 1 1 3 1 1 0 bad
446 3 9 4 0 0 0 0 1 0 1244 4 4 4 0 0 0 0 0 3 0 0 41 0 1 0 2 1 1 0 0 good
447 0 36 2 1 0 0 0 0 0 1842 0 1 4 0 0 0 0 0 3 0 0 34 0 0 1 1 2 1 1 0 bad
448 1 7 2 0 0 0 1 0 0 2576 0 2 2 0 1 0 0 1 1 1 0 35 0 0 1 1 2 1 0 0 good
449 2 12 2 0 0 1 0 0 0 1424 4 4 3 0 0 0 0 0 3 1 0 55 0 0 1 1 3 1 1 0 good
450 1 15 3 0 0 0 0 0 0 1512 3 2 3 0 0 1 0 0 2 0 0 61 1 0 1 2 2 1 0 0 bad
451 3 36 4 0 1 0 0 0 0 11054 4 2 4 0 1 0 0 0 1 0 0 30 0 0 1 1 3 1 1 0 good
452 3 6 2 0 0 0 1 0 0 518 0 2 3 0 0 0 0 0 0 1 0 29 0 0 1 1 2 1 0 0 good
453 3 12 0 0 0 1 0 0 0 2759 0 4 2 0 1 0 0 0 3 0 0 34 0 0 1 2 2 1 0 0 good
454 3 24 2 0 1 0 0 0 0 2670 0 4 4 0 1 0 0 0 3 0 0 35 0 0 1 1 3 1 1 0 good
455 0 24 2 1 0 0 0 0 0 4817 0 3 2 0 1 0 1 0 2 0 0 31 0 0 1 1 2 1 1 0 bad
456 3 24 2 0 1 0 0 0 0 2679 0 1 4 0 0 0 0 0 0 0 1 29 0 0 1 1 3 1 1 0 good
457 0 11 4 1 0 0 0 0 0 3905 0 2 2 0 1 0 0 0 1 1 0 36 0 1 0 2 2 2 0 0 good
458 0 12 2 0 1 0 0 0 0 3386 0 4 3 0 1 0 0 0 3 0 1 35 0 0 0 1 2 1 1 0 bad
459 0 6 2 0 0 0 0 0 0 343 0 1 4 0 0 0 0 0 0 1 0 27 0 0 1 1 2 1 0 0 good
460 3 18 2 0 0 0 1 0 0 4594 0 1 3 0 1 0 0 0 1 0 0 32 0 0 1 1 2 1 1 0 good
461 0 36 2 0 0 1 0 0 0 3620 0 2 1 0 1 0 0 1 1 0 0 37 0 0 1 1 2 2 0 0 good
462 0 15 2 1 0 0 0 0 0 1721 0 1 2 0 1 0 0 0 2 1 0 36 0 0 1 1 2 1 0 0 good
463 1 12 2 0 0 1 0 0 0 3017 0 1 3 0 0 0 0 0 0 1 0 34 0 1 0 1 3 1 0 0 good
464 1 12 2 0 0 0 0 0 0 754 4 4 4 0 1 0 0 0 3 0 0 38 0 0 1 2 2 1 0 0 good
465 3 18 2 0 0 0 0 0 1 1950 0 3 4 0 1 0 0 0 0 0 0 34 1 0 1 2 2 1 1 0 good
466 0 24 2 0 1 0 0 0 0 2924 0 2 3 0 1 0 0 1 3 0 1 63 1 0 1 1 2 2 1 0 good
467 0 24 3 0 0 0 1 0 0 1659 0 1 4 0 0 0 0 0 1 0 0 29 0 1 0 1 1 1 1 0 bad
468 3 48 3 0 0 0 1 0 0 7238 4 4 3 0 1 0 0 0 2 0 0 32 1 0 1 2 2 2 0 0 good
469 3 33 3 0 0 0 0 0 1 2764 0 2 2 0 0 0 0 0 1 0 0 26 0 0 1 2 2 1 1 0 good
470 3 24 3 0 1 0 0 0 0 4679 0 3 3 0 1 0 0 0 2 0 0 35 0 0 1 2 1 1 1 0 good
471 1 24 2 0 0 0 1 0 0 3092 1 1 3 0 0 1 0 0 1 0 0 22 0 1 0 1 2 1 1 0 bad
472 0 6 2 0 0 0 0 1 0 448 0 1 4 0 0 0 0 0 3 0 0 23 0 0 1 1 2 1 0 0 bad
473 0 9 2 1 0 0 0 0 0 654 0 2 4 0 1 0 0 0 2 0 0 28 0 0 1 1 1 1 0 0 bad
474 3 6 2 0 0 0 0 0 0 1238 4 0 4 0 1 0 0 0 3 0 0 36 0 0 1 1 3 2 1 0 good
475 1 18 4 0 0 0 1 0 0 1245 0 2 4 0 0 1 0 0 1 0 0 33 0 0 1 1 2 1 0 0 bad
476 0 18 0 0 0 1 0 0 0 3114 0 1 1 0 0 0 0 0 3 0 0 26 0 1 0 1 2 1 0 0 bad
477 3 39 2 0 1 0 0 0 0 2569 2 2 4 0 1 0 0 0 3 0 0 24 0 0 1 1 2 1 0 0 good
478 2 24 2 0 0 0 1 0 0 5152 0 3 4 0 1 0 0 0 1 0 0 25 1 0 1 1 2 1 0 0 good
479 1 12 2 0 0 0 0 0 1 1037 1 3 3 0 1 0 0 0 3 1 0 39 0 0 1 1 1 1 0 0 good
480 0 15 4 0 0 1 0 0 0 1478 0 4 4 0 1 0 0 0 3 0 0 44 0 0 1 2 2 2 1 0 good
481 1 12 4 0 0 0 1 0 0 3573 0 2 1 0 0 0 0 0 0 1 0 23 0 0 1 1 1 1 0 0 good
482 1 24 2 1 0 0 0 0 0 1201 0 1 4 0 1 0 0 0 0 0 0 26 0 0 1 1 2 1 0 0 good
483 0 30 2 0 0 1 0 0 0 3622 3 4 4 0 0 0 0 0 3 0 0 57 0 1 0 2 2 1 1 0 good
484 3 15 3 0 0 1 0 0 0 960 3 3 3 0 0 0 0 0 1 0 0 30 0 0 1 2 2 1 0 0 good
485 3 12 4 1 0 0 0 0 0 1163 2 2 4 0 1 0 0 0 3 1 0 44 0 0 1 1 2 1 1 0 good
486 1 6 3 1 0 0 0 0 0 1209 0 0 4 0 1 0 0 0 3 0 0 47 0 0 1 1 3 1 1 0 bad
487 3 12 2 0 0 0 1 0 0 3077 0 2 2 0 1 0 0 0 3 0 0 52 0 0 1 1 2 1 1 0 good
488 3 24 2 1 0 0 0 0 0 3757 0 4 4 0 0 0 1 0 3 0 1 62 0 0 0 1 2 1 1 0 good
489 3 10 2 1 0 0 0 0 0 1418 1 2 3 0 1 0 0 0 1 1 0 35 0 1 0 1 1 1 0 1 good
490 3 6 2 1 0 0 0 0 0 3518 0 2 2 0 1 0 0 1 2 0 0 26 0 1 0 1 2 1 0 0 good
491 3 12 4 0 0 0 1 0 0 1934 0 4 2 0 1 0 0 0 1 0 1 26 0 0 1 2 2 1 0 0 good
492 1 27 0 0 0 0 0 0 1 8318 0 4 2 0 0 0 0 0 3 0 1 42 0 0 0 2 3 1 1 0 bad
493 3 6 4 0 0 0 1 0 0 1237 1 2 1 0 0 0 0 0 0 0 0 27 0 0 1 2 2 1 0 0 good
494 1 6 2 0 0 0 1 0 0 368 4 4 4 0 1 0 0 0 3 0 0 38 0 0 1 1 2 1 0 0 good
495 0 12 4 1 0 0 0 0 0 2122 0 2 3 0 1 0 0 0 1 1 0 39 0 1 0 2 1 2 0 1 good
496 0 24 2 0 0 1 0 0 0 2996 4 2 2 0 0 1 0 0 3 0 0 20 0 0 1 1 2 1 0 0 bad
497 1 36 2 0 0 1 0 0 0 9034 1 1 4 0 1 0 1 0 0 0 1 29 0 1 0 1 3 1 1 0 bad
498 3 24 4 0 0 1 0 0 0 1585 0 3 4 0 1 0 0 0 2 0 0 40 0 0 1 2 2 1 0 0 good
499 1 18 2 0 0 0 1 0 0 1301 0 4 4 0 0 1 0 1 1 1 0 32 0 0 1 1 1 1 0 0 good
500 2 6 4 1 0 0 0 0 0 1323 1 4 2 1 0 0 0 0 3 0 0 28 0 0 1 2 2 2 1 0 good
501 0 24 2 1 0 0 0 0 0 3123 0 1 4 0 0 0 0 0 0 0 0 27 0 0 1 1 2 1 0 0 bad
502 0 36 2 0 1 0 0 0 0 5493 0 4 2 0 1 0 0 0 3 0 1 42 0 0 0 1 2 2 0 0 good
503 2 9 2 0 0 0 1 0 0 1126 1 4 2 1 0 0 0 0 3 1 0 49 0 0 1 1 2 1 0 0 good
504 1 24 4 0 0 0 1 0 0 1216 1 1 4 0 1 0 0 0 3 0 1 38 1 0 1 2 2 2 0 0 bad
505 0 24 2 1 0 0 0 0 0 1207 0 1 4 0 0 0 0 0 3 0 0 24 0 1 0 1 2 1 0 0 bad
506 3 10 2 1 0 0 0 0 0 1309 4 2 4 0 1 0 0 1 3 0 0 27 0 0 1 1 1 1 0 0 bad
507 2 15 4 0 1 0 0 0 0 2360 2 2 2 0 1 0 0 0 1 0 0 36 0 0 1 1 2 1 1 0 good
508 1 15 1 1 0 0 0 0 0 6850 1 0 1 0 1 0 0 0 1 0 0 34 0 0 1 1 3 2 1 0 bad
509 3 24 2 0 0 0 1 0 0 1413 0 2 4 0 0 1 0 0 1 0 0 28 0 0 1 1 2 1 0 0 good
510 3 39 2 0 1 0 0 0 0 8588 1 4 4 0 1 0 0 0 1 0 0 45 0 0 1 1 3 1 1 0 good
511 0 12 2 1 0 0 0 0 0 759 0 3 4 0 1 0 0 0 1 1 0 26 0 0 1 1 2 1 0 0 bad
512 3 36 2 0 1 0 0 0 0 4686 0 2 2 0 1 0 0 0 1 0 1 32 0 0 0 1 3 1 1 0 good
513 2 15 2 0 0 0 0 0 1 2687 0 3 2 0 1 0 0 0 3 0 0 26 0 1 0 1 2 1 1 0 good
514 1 12 3 0 0 0 1 0 0 585 0 2 4 0 0 1 1 0 3 1 0 20 0 1 0 2 2 1 0 0 good
515 3 24 2 1 0 0 0 0 0 2255 4 1 4 0 1 0 0 0 0 0 0 54 0 0 1 1 2 1 0 0 good
516 0 6 4 1 0 0 0 0 0 609 0 3 4 0 0 0 0 0 2 0 0 37 0 0 1 2 2 1 0 1 good
517 0 6 4 1 0 0 0 0 0 1361 0 1 2 0 1 0 0 0 3 1 0 40 0 0 1 1 1 2 0 1 good
518 3 36 4 0 0 1 0 0 0 7127 0 1 2 0 0 0 0 0 3 0 0 23 0 1 0 2 2 1 1 0 bad
519 0 6 2 1 0 0 0 0 0 1203 1 4 3 0 1 0 0 0 1 0 0 43 0 0 1 1 2 1 1 0 good
520 3 6 4 0 0 0 1 0 0 700 4 4 4 0 1 0 0 0 3 0 1 36 0 0 0 2 2 1 0 0 good
521 3 24 4 0 0 0 0 0 0 5507 0 4 3 0 1 0 0 0 3 0 1 44 0 0 0 2 2 1 0 0 good
522 0 18 2 0 0 0 1 0 0 3190 0 2 2 0 0 0 0 0 1 1 0 24 0 0 1 1 2 1 0 0 bad
523 0 48 0 0 0 1 0 0 0 7119 0 2 3 0 1 0 0 0 3 0 1 53 0 0 0 2 2 2 0 0 bad
524 3 24 2 0 1 0 0 0 0 3488 1 3 3 0 0 0 0 0 3 0 0 23 0 0 1 1 2 1 0 0 good
525 1 18 2 0 0 0 1 0 0 1113 0 2 4 0 0 0 0 1 3 1 0 26 0 0 1 1 1 2 0 0 good
526 1 26 2 0 1 0 0 0 0 7966 0 1 2 0 1 0 0 0 2 0 0 30 0 0 1 2 2 1 0 0 good
527 3 15 4 0 0 0 0 1 0 1532 1 2 4 0 0 0 0 0 2 0 0 31 0 0 1 1 2 1 0 0 good
528 3 4 4 0 0 0 1 0 0 1503 0 3 2 0 1 0 0 0 0 1 0 42 0 0 1 2 1 2 0 0 good
529 0 36 2 0 0 0 1 0 0 2302 0 2 4 1 0 0 0 0 3 0 0 31 0 1 0 1 2 1 0 0 bad
530 0 6 2 1 0 0 0 0 0 662 0 1 3 0 1 0 0 0 3 1 0 41 0 0 1 1 1 2 1 0 good
531 1 36 2 0 0 0 0 1 0 2273 0 3 3 0 1 0 0 0 0 0 0 32 0 0 1 2 2 2 0 0 good
532 1 15 2 1 0 0 0 0 0 2631 1 2 2 0 0 0 0 0 3 0 0 28 0 1 0 2 2 1 1 0 bad
533 3 12 3 0 1 0 0 0 0 1503 0 2 4 0 0 1 0 0 3 1 0 41 0 1 0 1 2 1 0 0 good
534 3 24 2 0 0 0 1 0 0 1311 1 3 4 0 0 1 0 0 2 0 0 26 0 0 1 1 2 1 1 0 good
535 3 24 2 0 0 0 1 0 0 3105 4 1 4 0 1 0 0 0 1 0 0 25 0 0 1 2 2 1 0 0 good
536 2 21 4 0 0 0 0 1 0 2319 0 1 2 1 0 0 0 0 0 0 0 33 0 1 0 1 2 1 0 0 bad
537 0 6 2 1 0 0 0 0 0 1374 4 0 4 0 0 0 0 0 2 0 0 33 0 0 1 1 3 1 1 0 good
538 1 18 4 0 0 1 0 0 0 3612 0 4 3 0 0 0 0 0 3 0 0 37 0 0 1 1 2 1 1 0 good
539 0 48 2 1 0 0 0 0 0 7763 0 4 4 0 1 0 0 0 3 0 1 42 1 0 0 1 3 1 0 0 bad
540 2 18 2 0 0 1 0 0 0 3049 0 1 1 0 0 0 0 0 0 0 0 45 1 0 1 1 1 1 0 0 good
541 1 12 2 0 0 0 1 0 0 1534 0 1 1 0 0 1 0 0 0 1 0 23 0 1 0 1 2 1 0 0 bad
542 3 24 3 1 0 0 0 0 0 2032 0 4 4 0 1 0 0 0 3 0 1 60 0 0 0 2 2 1 1 0 good
543 0 30 2 0 0 1 0 0 0 6350 4 4 4 0 1 0 0 0 3 0 0 31 0 0 1 1 2 1 0 0 bad
544 2 18 2 0 0 1 0 0 0 2864 0 2 2 0 1 0 0 0 0 1 0 34 0 0 1 1 1 2 0 0 bad
545 3 12 4 1 0 0 0 0 0 1255 0 4 4 0 1 0 0 0 3 1 0 61 0 0 1 2 1 1 0 0 good
546 0 24 3 1 0 0 0 0 0 1333 0 0 4 0 1 0 0 0 1 1 0 43 0 0 0 2 2 2 0 0 bad
547 3 24 4 1 0 0 0 0 0 2022 0 2 4 0 0 0 0 0 3 0 0 37 0 0 1 1 2 1 1 0 good
548 3 24 2 0 0 0 1 0 0 1552 0 3 3 0 1 0 0 0 0 0 0 32 1 0 1 1 2 2 0 0 good
549 0 12 1 0 0 0 1 0 0 626 0 2 4 0 0 0 0 0 3 1 0 24 1 0 1 1 1 1 0 0 bad
550 3 48 4 0 1 0 0 0 0 8858 4 3 2 0 1 0 0 0 0 0 1 35 0 0 0 2 2 1 1 0 good
551 3 12 4 0 0 0 0 0 0 996 4 3 4 0 0 0 0 0 3 1 0 23 0 0 1 2 2 1 0 0 good
552 3 6 1 0 0 0 1 0 0 1750 2 4 2 0 1 0 0 0 3 0 0 45 1 0 1 1 1 2 0 0 good
553 0 48 2 0 0 0 1 0 0 6999 0 3 1 0 0 1 0 1 0 1 0 34 0 0 1 2 2 1 1 0 bad
554 1 12 4 1 0 0 0 0 0 1995 1 1 4 0 1 0 0 0 0 0 0 27 0 0 1 1 2 1 0 0 good
555 1 9 2 0 0 0 0 1 0 1199 0 3 4 0 0 0 0 0 3 0 0 67 0 0 1 2 3 1 1 0 good
556 1 12 2 0 0 0 1 0 0 1331 0 1 2 0 1 0 0 0 0 0 0 22 1 0 1 1 2 1 0 0 bad
557 1 18 0 1 0 0 0 0 0 2278 1 1 3 0 0 0 0 0 2 0 0 28 0 0 1 2 2 1 0 0 bad
558 3 21 0 1 0 0 0 0 0 5003 4 2 1 0 0 0 0 0 3 0 0 29 1 0 1 2 2 1 1 0 bad
559 0 24 1 0 0 1 0 0 0 3552 0 3 3 0 1 0 0 0 3 0 0 27 1 0 1 1 2 1 0 0 bad
560 1 18 4 0 0 1 0 0 0 1928 0 1 2 0 1 0 0 0 1 1 0 31 0 0 1 2 1 1 0 0 bad
561 0 24 2 0 1 0 0 0 0 2964 4 4 4 0 1 0 0 0 3 0 1 49 1 0 0 1 2 2 1 0 good
562 0 24 1 0 0 0 1 0 0 1546 0 3 4 0 1 0 0 1 3 0 0 24 1 1 0 1 1 1 0 0 bad
563 2 6 3 0 0 0 1 0 0 683 0 1 2 0 0 0 0 0 0 0 0 29 1 0 1 1 2 1 0 0 good
564 1 36 2 1 0 0 0 0 0 12389 4 2 1 0 1 0 0 0 3 0 1 37 0 0 0 1 2 1 1 0 bad
565 1 24 3 0 0 0 0 0 1 4712 4 2 4 0 1 0 0 0 1 0 0 37 1 0 1 2 3 1 1 0 good
566 1 24 3 0 0 0 1 0 0 1553 1 3 3 0 0 0 0 0 1 0 0 23 0 1 0 2 2 1 1 0 good
567 0 12 2 1 0 0 0 0 0 1372 0 3 2 1 0 0 0 0 2 0 0 36 0 0 1 1 2 1 0 0 bad
568 3 24 4 0 0 0 1 0 0 2578 3 4 2 0 1 0 0 0 1 0 0 34 0 0 1 1 2 1 0 0 good
569 1 48 2 0 0 0 1 0 0 3979 4 3 4 0 1 0 0 0 0 0 0 41 0 0 1 2 2 2 1 0 good
570 0 48 2 0 0 0 1 0 0 6758 0 2 3 0 0 0 0 0 1 0 0 31 0 0 1 1 2 1 1 0 bad
571 0 24 2 0 0 1 0 0 0 3234 0 1 4 0 0 0 0 0 3 1 0 23 0 1 0 1 1 1 1 0 bad
572 3 30 4 0 0 0 1 0 0 5954 0 3 3 0 1 0 1 0 1 0 0 38 0 0 1 1 2 1 0 0 good
573 3 24 2 0 1 0 0 0 0 5433 4 0 2 0 0 0 0 0 3 0 0 26 0 1 0 1 3 1 1 0 good
574 0 15 2 0 0 0 0 0 1 806 0 2 4 0 0 0 0 0 3 0 0 22 0 0 1 1 1 1 0 0 good
575 1 9 2 0 0 0 1 0 0 1082 0 4 4 0 1 0 0 0 3 0 0 27 0 0 1 2 1 1 0 0 good
576 3 15 4 0 0 1 0 0 0 2788 0 3 2 0 0 0 1 0 2 0 0 24 1 0 1 2 2 1 0 0 good
577 1 12 2 0 0 0 1 0 0 2930 0 3 2 0 0 0 0 0 0 1 0 27 0 0 1 1 2 1 0 0 good
578 3 24 4 0 0 0 0 1 0 1927 4 2 3 0 0 0 0 0 1 0 0 33 0 0 1 2 2 1 1 0 good
579 1 36 4 1 0 0 0 0 0 2820 0 1 4 1 0 0 0 0 3 0 0 27 0 0 1 2 2 1 0 0 bad
580 3 24 2 0 0 0 0 0 0 937 0 1 4 0 0 1 0 0 2 0 0 27 0 0 1 2 1 1 0 0 good
581 1 18 4 1 0 0 0 0 0 1056 0 4 3 0 1 0 0 1 2 1 0 30 1 0 1 2 2 1 0 0 bad
582 1 12 4 1 0 0 0 0 0 3124 0 1 1 0 1 0 0 0 2 1 0 49 1 0 1 2 1 2 0 0 good
583 3 9 2 0 0 1 0 0 0 1388 0 2 4 0 0 0 0 0 1 1 0 26 0 1 0 1 2 1 0 0 good
584 1 36 2 0 0 0 0 0 0 2384 0 1 4 0 1 0 0 0 0 0 1 33 0 1 0 1 1 1 0 0 bad
585 3 12 2 1 0 0 0 0 0 2133 4 4 4 0 0 0 0 0 3 0 1 52 0 0 0 1 3 1 1 0 good
586 0 18 2 0 0 1 0 0 0 2039 0 2 1 0 0 0 0 0 3 1 0 20 1 1 0 1 2 1 0 0 bad
587 0 9 4 1 0 0 0 0 0 2799 0 2 2 0 1 0 0 0 1 1 0 36 0 1 0 2 2 2 0 0 good
588 0 12 2 0 0 1 0 0 0 1289 0 2 4 0 1 0 0 1 0 0 0 21 0 0 1 1 1 1 0 0 good
589 0 18 2 0 0 0 0 0 0 1217 0 2 4 0 0 1 0 0 2 1 0 47 0 0 1 1 1 1 1 0 bad
590 0 12 4 0 0 1 0 0 0 2246 0 4 3 0 1 0 0 0 2 0 0 60 0 0 1 2 2 1 0 0 bad
591 0 12 4 0 0 0 1 0 0 385 0 3 4 0 0 0 0 0 2 1 0 58 0 0 1 4 1 1 1 0 good
592 1 24 3 1 0 0 0 0 0 1965 4 2 4 0 0 0 0 0 3 0 0 42 0 1 0 2 2 1 1 0 good
593 3 21 2 0 0 0 0 0 1 1572 3 4 4 0 0 0 0 0 3 1 0 36 1 0 1 1 1 1 0 0 good
594 1 24 2 1 0 0 0 0 0 2718 0 2 3 0 0 0 0 0 3 0 0 20 0 1 0 1 1 1 1 0 bad
595 0 24 1 0 0 0 0 0 0 1358 4 4 4 0 1 0 0 0 2 0 0 40 1 0 1 1 3 1 1 0 bad
596 1 6 1 1 0 0 0 0 0 931 1 1 1 0 0 0 0 0 0 0 0 32 1 0 1 1 1 1 0 0 bad
597 0 24 2 1 0 0 0 0 0 1442 0 3 4 0 0 0 0 0 3 0 0 23 0 1 0 2 2 1 0 0 bad
598 1 24 0 0 0 0 0 0 1 4241 0 2 1 0 1 0 0 0 3 1 0 36 0 0 1 3 1 1 1 0 bad
599 3 18 4 1 0 0 0 0 0 2775 0 3 2 0 1 0 0 0 1 0 0 31 1 0 1 2 2 1 0 0 bad
600 3 24 3 0 0 0 0 0 1 3863 0 2 1 0 1 0 0 0 1 0 1 32 0 0 0 1 2 1 0 0 good
601 1 7 2 0 0 0 1 0 0 2329 0 1 1 0 0 0 0 1 0 1 0 45 0 0 1 1 2 1 0 0 good
602 1 9 2 0 0 1 0 0 0 918 0 2 4 0 0 0 0 0 0 0 0 30 0 0 1 1 2 1 0 0 bad
603 1 24 1 0 0 0 0 1 0 1837 0 3 4 0 0 0 0 0 3 0 1 34 1 0 0 1 1 1 0 0 bad
604 3 36 2 0 0 1 0 0 0 3349 0 2 4 0 0 0 0 0 1 0 0 28 0 0 1 1 3 1 1 0 bad
605 2 10 2 0 0 1 0 0 0 1275 0 1 4 0 0 0 0 0 1 0 0 23 0 0 1 1 2 1 0 0 good
606 0 24 1 0 0 1 0 0 0 2828 2 2 4 0 1 0 0 0 3 1 0 22 1 0 1 1 2 1 1 0 good
607 3 24 4 0 0 0 0 0 1 4526 0 2 3 0 1 0 0 0 1 1 0 74 0 0 1 1 3 1 1 0 good
608 1 36 2 0 0 0 1 0 0 2671 1 2 4 0 0 0 1 0 3 0 1 50 0 0 0 1 2 1 0 0 bad
609 3 18 2 0 0 0 1 0 0 2051 0 1 4 0 1 0 0 0 0 1 0 33 0 0 1 1 2 1 0 0 good
610 3 15 2 0 1 0 0 0 0 1300 4 4 4 0 1 0 0 0 3 0 1 45 1 0 0 1 2 2 0 0 good
611 0 12 2 0 0 0 0 0 0 741 1 0 4 0 0 0 0 0 2 0 0 22 0 0 1 1 2 1 0 0 bad
612 2 10 2 1 0 0 0 0 0 1240 1 4 1 0 0 0 0 0 3 0 1 48 0 0 0 1 1 2 0 0 bad
613 0 21 2 0 0 0 1 0 0 3357 3 1 4 0 0 0 0 0 1 0 0 29 1 0 1 1 2 1 0 0 good
614 0 24 1 0 1 0 0 0 0 3632 0 2 1 0 0 0 0 1 3 0 0 22 1 1 0 1 2 1 0 1 good
615 3 18 3 0 0 1 0 0 0 1808 0 3 4 0 0 0 0 0 0 1 0 22 0 0 1 1 2 1 0 0 bad
616 1 48 0 0 0 0 0 0 1 12204 4 2 2 0 1 0 0 0 1 0 0 48 1 0 1 1 3 1 1 0 good
617 1 60 3 0 0 0 1 0 0 9157 4 2 2 0 1 0 0 0 1 0 1 27 0 0 0 1 3 1 0 0 good
618 0 6 4 1 0 0 0 0 0 3676 0 2 1 0 1 0 0 0 2 1 0 37 0 1 0 3 2 2 0 0 good
619 1 30 2 0 0 1 0 0 0 3441 1 2 2 0 0 0 1 0 3 0 0 21 0 1 0 1 2 1 0 0 bad
620 3 12 2 1 0 0 0 0 0 640 0 2 4 1 0 0 0 0 1 1 0 49 0 0 1 1 1 1 0 0 good
621 1 21 4 0 0 0 0 0 1 3652 0 3 2 0 1 0 0 0 2 0 0 27 0 0 1 2 2 1 0 0 good
622 3 18 4 1 0 0 0 0 0 1530 0 2 3 0 1 0 0 0 1 0 0 32 1 0 1 2 2 1 0 0 bad
623 3 48 2 0 0 0 0 0 1 3914 4 2 4 1 0 0 0 0 1 1 0 38 1 0 1 1 2 1 0 0 bad
624 0 12 2 0 0 1 0 0 0 1858 0 1 4 0 0 0 0 0 0 0 0 22 0 1 0 1 2 1 0 0 good
625 0 18 2 0 0 0 1 0 0 2600 0 2 4 0 1 0 0 0 3 0 1 65 0 0 0 2 2 1 0 0 bad
626 3 15 2 0 0 0 1 0 0 1979 4 4 4 0 1 0 0 0 1 0 0 35 0 0 1 1 2 1 0 0 good
627 2 6 2 0 0 1 0 0 0 2116 0 2 2 0 1 0 0 0 1 1 0 41 0 0 1 1 2 1 1 0 good
628 1 9 1 1 0 0 0 0 0 1437 1 3 2 0 1 0 0 0 2 0 1 29 0 0 1 1 2 1 0 0 bad
629 3 42 4 0 0 1 0 0 0 4042 2 2 4 0 1 0 0 0 3 1 0 36 0 0 1 2 2 1 1 0 good
630 3 9 2 0 0 0 0 1 0 3832 4 4 1 0 1 0 0 0 3 1 0 64 0 0 1 1 1 1 0 0 good
631 0 24 2 0 0 0 1 0 0 3660 0 2 2 0 0 0 0 0 3 0 0 28 0 0 1 1 2 1 0 0 good
632 0 18 1 0 0 1 0 0 0 1553 0 2 4 0 1 0 0 0 2 0 0 44 1 0 1 1 2 1 0 0 bad
633 1 15 2 0 0 0 1 0 0 1444 4 1 4 0 1 0 0 0 0 0 0 23 0 0 1 1 2 1 0 0 good
634 3 9 2 0 0 1 0 0 0 1980 0 1 2 0 0 0 1 0 1 0 0 19 0 1 0 2 2 1 0 0 bad
635 1 24 2 1 0 0 0 0 0 1355 0 1 3 0 0 0 0 0 3 0 0 25 0 0 1 1 1 1 1 0 bad
636 3 12 2 0 0 0 0 1 0 1393 0 4 4 0 1 0 0 0 3 0 0 47 1 0 1 3 2 2 1 0 good
637 3 24 2 0 0 0 1 0 0 1376 2 3 4 0 0 0 0 0 0 0 0 28 0 0 1 1 2 1 0 0 good
638 3 60 3 0 0 0 1 0 0 15653 0 3 2 0 1 0 0 0 3 0 0 21 0 0 1 2 2 1 1 0 good
639 3 12 2 0 0 0 1 0 0 1493 0 1 4 0 0 0 0 0 2 0 0 34 0 0 1 1 2 2 0 0 good
640 0 42 3 0 0 0 1 0 0 4370 0 3 3 0 1 0 0 0 1 0 0 26 1 0 1 2 2 2 1 0 bad
641 0 18 2 0 0 0 0 1 0 750 0 0 4 0 0 0 0 0 0 1 0 27 0 0 1 1 0 1 0 0 bad
642 1 15 2 0 0 0 0 0 0 1308 0 4 4 0 1 0 0 0 3 0 0 38 0 0 1 2 1 1 0 0 good
643 3 15 2 0 0 0 0 1 0 4623 1 2 3 0 1 0 0 0 1 0 0 40 0 0 1 1 3 1 1 0 bad
644 3 24 4 0 0 0 1 0 0 1851 0 3 4 0 0 1 0 1 1 0 0 33 0 0 1 2 2 1 1 0 good
645 0 18 4 0 0 0 1 0 0 1880 0 3 4 0 0 1 0 0 0 0 0 32 0 0 1 2 3 1 1 0 good
646 3 36 3 0 0 0 0 0 1 7980 4 1 4 0 1 0 0 0 3 0 0 27 0 1 0 2 2 1 1 0 bad
647 0 30 0 0 0 1 0 0 0 4583 0 2 2 1 0 0 0 1 1 1 0 32 0 0 1 2 2 1 0 0 good
648 3 12 2 1 0 0 0 0 0 1386 2 2 2 0 0 0 0 0 1 0 0 26 0 0 1 1 2 1 0 0 bad
649 2 24 2 1 0 0 0 0 0 947 0 3 4 0 1 0 0 0 2 0 1 38 1 0 0 1 2 2 0 0 bad
650 0 12 2 0 0 0 0 1 0 684 0 2 4 0 1 0 0 0 3 0 0 40 0 1 0 1 1 2 0 0 bad
651 0 48 2 0 0 0 0 1 0 7476 0 3 4 0 1 0 0 0 0 0 1 50 0 0 0 1 3 1 1 0 good
652 1 12 2 0 0 1 0 0 0 1922 0 2 4 0 1 0 0 0 1 0 0 37 0 0 1 1 1 1 0 0 bad
653 0 24 2 1 0 0 0 0 0 2303 0 4 4 0 1 0 1 0 0 1 0 45 0 0 1 1 2 1 0 0 bad
654 1 36 3 1 0 0 0 0 0 8086 1 4 2 0 1 0 0 0 3 0 0 42 0 0 1 4 3 1 1 0 bad
655 3 24 4 0 1 0 0 0 0 2346 0 3 4 0 1 0 0 0 2 0 0 35 0 0 1 2 2 1 1 0 good
656 0 14 2 1 0 0 0 0 0 3973 0 0 1 0 1 0 0 0 3 0 1 22 0 0 0 1 2 1 0 0 good
657 1 12 2 1 0 0 0 0 0 888 0 4 4 0 1 0 0 0 3 0 0 41 1 0 1 1 1 2 0 0 bad
658 3 48 2 0 0 0 1 0 0 10222 4 3 4 0 1 0 0 0 2 0 0 37 1 0 1 1 2 1 1 0 good
659 1 30 0 0 0 0 0 0 1 4221 0 2 2 0 0 0 0 0 0 0 0 28 0 0 1 2 2 1 0 0 good
660 1 18 4 0 0 1 0 0 0 6361 0 4 2 0 1 0 0 0 0 0 1 41 0 0 1 1 2 1 1 0 good
661 2 12 2 0 0 0 1 0 0 1297 0 2 3 0 0 1 0 0 3 1 0 23 0 1 0 1 2 1 0 0 good
662 0 12 2 1 0 0 0 0 0 900 4 2 4 0 0 1 0 0 1 0 0 23 0 0 1 1 2 1 0 0 bad
663 3 21 2 0 0 1 0 0 0 2241 0 4 4 0 1 0 0 0 1 1 0 50 0 0 1 2 2 1 0 0 good
664 1 6 3 0 0 1 0 0 0 1050 0 0 4 0 1 0 0 0 0 0 0 35 1 0 1 2 3 1 1 0 good
665 2 6 4 0 0 0 0 1 0 1047 0 2 2 0 0 0 0 0 3 0 0 50 0 0 1 1 1 1 0 0 good
666 3 24 4 0 0 0 0 0 0 6314 0 0 4 0 1 0 1 0 1 0 1 27 1 0 1 2 3 1 1 0 good
667 1 30 1 0 0 1 0 0 0 3496 3 2 4 0 1 0 0 0 1 0 0 34 1 0 1 1 2 2 1 0 good
668 3 48 1 0 0 0 0 0 1 3609 0 2 1 0 0 0 0 0 0 1 0 27 1 0 1 1 2 1 0 0 good
669 0 12 4 1 0 0 0 0 0 4843 0 4 3 0 1 0 1 0 3 0 0 43 0 1 0 2 2 1 1 0 bad
670 2 30 4 0 0 0 1 0 0 3017 0 4 4 0 1 0 0 0 3 0 0 47 0 0 1 1 2 1 0 0 good
671 3 24 4 0 0 0 0 0 1 4139 1 2 3 0 1 0 0 0 2 0 0 27 0 0 1 2 1 1 1 0 good
672 3 36 2 0 0 0 0 0 1 5742 1 3 2 0 1 0 0 0 1 0 0 31 0 0 1 2 2 1 1 0 good
673 3 60 2 1 0 0 0 0 0 10366 0 4 2 0 1 0 0 0 3 0 0 42 0 0 1 1 3 1 1 0 good
674 3 6 4 1 0 0 0 0 0 2080 2 2 1 0 0 1 0 0 1 0 0 24 0 0 1 1 2 1 0 0 good
675 3 21 3 0 0 0 0 0 1 2580 2 1 4 0 1 0 0 0 1 1 0 41 1 0 1 1 1 2 0 0 bad
676 3 30 4 0 0 0 1 0 0 4530 0 3 4 0 0 0 0 0 3 0 0 26 0 1 0 1 3 1 1 0 good
677 3 24 4 0 0 1 0 0 0 5150 0 4 4 0 1 0 0 0 3 0 0 33 0 0 1 1 2 1 1 0 good
678 1 72 2 0 0 0 1 0 0 5595 1 2 2 0 0 1 0 0 1 0 0 24 0 0 1 1 2 1 0 0 bad
679 0 24 2 0 0 0 1 0 0 2384 0 4 4 0 1 0 0 0 3 1 0 64 1 1 0 1 1 1 0 0 good
680 3 18 2 0 0 0 1 0 0 1453 0 1 3 0 0 0 0 0 0 1 0 26 0 0 1 1 2 1 0 0 good
681 3 6 2 0 0 0 0 1 0 1538 0 1 1 0 0 0 0 0 1 0 1 56 0 0 1 1 2 1 0 0 good
682 3 12 2 0 0 0 1 0 0 2279 4 2 4 0 1 0 0 0 3 0 1 37 0 0 0 1 2 1 1 0 good
683 3 15 3 0 0 0 1 0 0 1478 0 2 4 0 0 1 0 0 2 1 0 33 1 0 1 2 2 1 0 0 good
684 3 24 4 0 0 0 1 0 0 5103 0 1 3 0 0 1 0 0 2 0 1 47 0 0 0 3 2 1 1 0 good
685 1 36 3 0 0 0 0 0 1 9857 1 3 1 0 1 0 0 0 2 0 0 31 0 0 1 2 1 2 1 0 good
686 3 60 2 1 0 0 0 0 0 6527 4 2 4 0 1 0 0 0 3 0 1 34 0 0 0 1 2 2 1 0 good
687 2 10 4 0 0 0 1 0 0 1347 4 3 4 0 1 0 0 0 1 0 0 27 0 0 1 2 2 1 1 0 good
688 1 36 3 1 0 0 0 0 0 2862 1 4 4 0 1 0 0 0 2 0 1 30 0 0 0 1 2 1 0 0 good
689 3 9 2 0 0 0 1 0 0 2753 1 4 3 0 1 0 1 0 3 0 0 35 0 0 1 1 2 1 1 0 good
690 0 12 2 1 0 0 0 0 0 3651 3 2 1 0 1 0 0 0 2 0 0 31 0 0 1 1 2 2 0 0 good
691 0 15 4 0 0 1 0 0 0 975 0 2 2 1 0 0 0 0 2 0 0 25 0 0 1 2 2 1 0 0 good
692 1 15 2 0 0 0 0 0 0 2631 1 2 3 0 0 0 0 0 1 1 0 25 0 0 1 1 1 1 0 0 good
693 1 24 2 0 0 0 1 0 0 2896 1 1 2 0 1 0 0 0 0 0 0 29 0 0 1 1 2 1 0 0 good
694 0 6 4 1 0 0 0 0 0 4716 4 1 1 0 1 0 0 0 2 1 0 44 0 0 1 2 1 2 0 0 good
695 3 24 2 0 0 0 1 0 0 2284 0 3 4 0 1 0 0 0 1 0 0 28 0 0 1 1 2 1 1 0 good
696 3 6 2 0 1 0 0 0 0 1236 2 2 2 0 1 0 0 0 3 0 0 50 0 1 0 1 2 1 0 0 good
697 1 12 2 0 0 0 1 0 0 1103 0 3 4 0 1 0 0 1 2 1 0 29 0 0 1 2 2 1 0 1 good
698 3 12 4 1 0 0 0 0 0 926 0 0 1 0 0 0 0 0 1 0 0 38 0 0 1 1 0 1 0 0 good
699 3 18 4 0 0 0 1 0 0 1800 0 2 4 0 1 0 0 0 1 0 0 24 0 0 1 2 2 1 0 0 good
700 2 15 2 0 0 0 0 1 0 1905 0 4 4 0 1 0 0 0 3 0 0 40 0 1 0 1 3 1 1 0 good
701 3 12 2 0 0 1 0 0 0 1123 2 2 4 0 0 0 0 0 3 0 0 29 0 1 0 1 1 1 0 0 bad
702 0 48 4 0 1 0 0 0 0 6331 0 4 4 0 1 0 0 0 3 0 1 46 0 0 0 2 2 1 1 0 bad
703 2 24 2 0 0 0 1 0 0 1377 1 4 4 0 0 0 0 0 1 0 1 47 0 0 0 1 2 1 1 0 good
704 1 30 3 0 0 0 0 0 1 2503 1 4 4 0 1 0 0 0 1 0 0 41 1 0 1 2 2 1 0 0 good
705 1 27 2 0 0 0 0 0 1 2528 0 1 4 0 0 0 0 0 0 0 0 32 0 0 1 1 2 2 1 0 good
706 3 15 2 1 0 0 0 0 0 5324 2 4 1 0 0 0 0 0 3 0 1 35 0 0 0 1 2 1 0 0 good
707 1 48 2 1 0 0 0 0 0 6560 1 3 3 0 1 0 0 0 1 0 0 24 0 0 1 1 2 1 0 0 bad
708 1 12 0 0 0 1 0 0 0 2969 0 1 4 0 0 0 0 0 2 0 0 25 0 1 0 2 2 1 0 0 bad
709 1 9 2 0 0 0 1 0 0 1206 0 4 4 0 0 0 0 0 3 1 0 25 0 0 1 1 2 1 0 0 good
710 1 9 2 0 0 0 1 0 0 2118 0 2 2 0 1 0 0 0 1 1 0 37 0 0 1 1 1 2 0 0 good
711 3 18 4 0 0 0 1 0 0 629 2 4 4 0 1 0 0 0 2 0 0 32 1 0 1 2 3 1 1 0 good
712 0 6 1 0 0 0 0 1 0 1198 0 4 4 0 0 0 0 0 3 0 1 35 0 0 0 1 2 1 0 0 bad
713 3 21 2 0 1 0 0 0 0 2476 4 4 4 0 1 0 0 0 3 1 0 46 0 0 1 1 3 1 1 0 good
714 0 9 4 0 0 0 1 0 0 1138 0 2 4 0 1 0 0 0 3 1 0 25 0 0 1 2 1 1 0 0 good
715 1 60 2 1 0 0 0 0 0 14027 0 3 4 0 1 0 0 0 1 0 1 27 0 0 1 1 3 1 1 0 bad
716 3 30 4 0 1 0 0 0 0 7596 4 4 1 0 1 0 0 0 3 0 0 63 0 0 1 2 2 1 0 0 good
717 3 30 4 0 0 0 1 0 0 3077 4 4 3 0 1 0 0 0 1 0 0 40 0 0 1 2 2 2 1 0 good
718 3 18 2 0 0 0 1 0 0 1505 0 2 4 0 1 0 0 0 1 0 1 32 0 0 0 1 3 1 1 0 good
719 2 24 4 0 0 0 1 0 0 3148 4 2 3 0 1 0 0 0 1 0 0 31 0 0 1 2 2 1 1 0 good
720 1 20 0 0 1 0 0 0 0 6148 1 4 3 0 0 1 0 0 3 0 0 31 1 0 1 2 2 1 1 0 good
721 2 9 0 0 0 0 1 0 0 1337 0 1 4 0 1 0 0 0 1 0 0 34 0 0 1 2 3 1 1 0 bad
722 1 6 1 0 0 0 0 1 0 433 3 1 4 0 0 0 0 0 1 0 0 24 1 1 0 1 2 2 0 0 bad
723 0 12 2 1 0 0 0 0 0 1228 0 2 4 0 0 0 0 0 1 1 0 24 0 0 1 1 1 1 0 0 bad
724 1 9 2 0 0 0 1 0 0 790 2 2 4 0 0 0 0 0 2 1 0 66 0 0 1 1 1 1 0 0 good
725 3 27 2 1 0 0 0 0 0 2570 0 2 3 0 0 0 0 0 2 1 0 21 0 1 0 1 2 1 0 0 bad
726 3 6 4 1 0 0 0 0 0 250 3 2 2 0 0 0 0 0 1 1 0 41 1 0 1 2 1 1 0 0 good
727 3 15 4 0 0 0 1 0 0 1316 2 2 2 0 0 1 0 0 1 0 0 47 0 0 1 2 1 1 0 0 good
728 0 18 2 0 0 0 1 0 0 1882 0 2 4 0 0 0 0 0 3 0 0 25 1 1 0 2 2 1 0 0 bad
729 1 48 1 0 0 0 0 0 1 6416 0 4 4 0 0 0 0 0 2 0 1 59 0 1 0 1 2 1 0 0 bad
730 2 24 4 0 0 0 0 0 1 1275 3 2 2 1 0 0 0 0 3 1 0 36 0 0 1 2 2 1 1 0 good
731 1 24 3 0 0 0 1 0 0 6403 0 1 1 0 1 0 0 0 1 0 0 33 0 0 1 1 2 1 0 0 good
732 0 24 2 0 0 0 1 0 0 1987 0 2 2 0 1 0 0 0 3 1 0 21 0 1 0 1 1 2 0 0 bad
733 1 8 2 0 0 0 1 0 0 760 0 3 4 0 0 0 0 1 1 1 0 44 0 0 1 1 1 1 0 0 good
734 3 24 2 0 1 0 0 0 0 2603 3 2 2 0 0 0 0 0 3 0 0 28 0 1 0 1 2 1 1 0 good
735 3 4 4 1 0 0 0 0 0 3380 0 3 1 0 0 0 0 0 0 1 0 37 0 0 1 1 2 2 0 0 good
736 1 36 1 0 0 0 0 0 0 3990 4 1 3 0 0 0 0 0 1 0 1 29 1 0 1 1 0 1 0 0 good
737 1 24 2 0 1 0 0 0 0 11560 0 2 1 0 0 0 0 0 3 0 0 23 0 1 0 2 3 1 0 0 bad
738 0 18 2 1 0 0 0 0 0 4380 1 2 3 0 1 0 0 0 3 0 0 35 0 0 1 1 1 2 1 0 good
739 3 6 4 1 0 0 0 0 0 6761 0 3 1 0 1 0 0 0 2 0 1 45 0 0 1 2 3 2 1 0 good
740 1 30 0 0 0 0 0 0 1 4280 1 2 4 0 0 0 0 0 3 0 0 26 0 1 0 2 1 1 0 0 bad
741 0 24 1 1 0 0 0 0 0 2325 1 3 2 0 1 0 0 0 2 0 0 32 1 0 1 1 2 1 0 0 good
742 1 10 1 0 0 0 1 0 0 1048 0 2 4 0 1 0 0 0 3 1 0 23 1 0 1 1 1 1 0 0 good
743 3 21 2 0 0 0 1 0 0 3160 4 4 4 0 1 0 0 0 2 0 0 41 0 0 1 1 2 1 1 0 good
744 0 24 1 0 0 1 0 0 0 2483 2 2 4 0 1 0 0 0 3 1 0 22 1 0 1 1 2 1 1 0 good
745 0 39 4 0 0 1 0 0 0 14179 4 3 4 0 1 0 0 0 3 0 0 30 0 0 1 2 3 1 1 0 good
746 0 13 4 0 0 0 0 0 1 1797 0 1 3 0 1 0 0 0 0 0 0 28 1 0 1 2 1 1 0 0 good
747 0 15 2 1 0 0 0 0 0 2511 0 0 1 0 0 0 0 0 3 0 0 23 0 1 0 1 2 1 0 0 good
748 0 12 2 1 0 0 0 0 0 1274 0 1 3 0 0 0 0 0 0 1 0 37 0 0 1 1 1 1 0 0 bad
749 3 21 2 0 1 0 0 0 0 5248 4 2 1 0 1 0 0 0 2 0 0 26 0 0 1 1 2 1 0 0 good
750 3 15 2 0 1 0 0 0 0 3029 0 3 2 0 1 0 0 0 1 0 0 33 0 0 1 1 2 1 0 0 good
751 0 6 2 0 0 1 0 0 0 428 0 4 2 0 0 0 0 0 0 0 0 49 1 0 1 1 2 1 1 0 good
752 0 18 2 1 0 0 0 0 0 976 0 1 1 0 0 0 0 0 1 0 0 23 0 0 1 1 1 1 0 0 bad
753 1 12 2 0 0 0 0 0 1 841 1 3 2 0 0 0 0 0 3 1 0 23 0 1 0 1 1 1 0 0 good
754 3 30 4 0 0 0 1 0 0 5771 0 3 4 0 0 0 0 0 1 0 0 25 0 0 1 2 2 1 0 0 good
755 3 12 3 0 0 0 0 0 0 1555 3 4 4 0 1 0 0 0 3 0 1 55 0 0 0 2 2 2 0 0 bad
756 0 24 2 1 0 0 0 0 0 1285 4 3 4 0 0 0 0 0 3 0 1 32 0 1 0 1 2 1 0 0 bad
757 2 6 4 1 0 0 0 0 0 1299 0 2 1 0 1 0 0 0 0 1 0 74 0 0 1 3 0 2 0 1 good
758 2 15 4 0 0 0 1 0 0 1271 4 2 3 0 1 0 0 0 3 0 1 39 0 0 0 2 2 1 1 0 bad
759 3 24 2 1 0 0 0 0 0 1393 0 2 2 0 1 0 0 1 1 1 0 31 0 0 1 1 2 1 1 0 good
760 0 12 4 1 0 0 0 0 0 691 0 4 4 0 1 0 0 0 2 0 0 35 0 0 1 2 2 1 0 0 bad
761 3 15 4 1 0 0 0 0 0 5045 4 4 1 0 0 0 0 0 3 0 0 59 0 0 1 1 2 1 1 0 good
762 0 18 4 0 0 1 0 0 0 2124 0 2 4 0 0 0 0 0 3 1 0 24 0 1 0 2 2 1 0 0 bad
763 0 12 2 0 0 0 1 0 0 2214 0 2 4 0 1 0 0 0 2 0 0 24 0 0 1 1 1 1 0 0 good
764 3 21 4 1 0 0 0 0 0 12680 4 4 4 0 1 0 0 0 3 0 1 30 0 0 0 1 3 1 1 0 bad
765 3 24 4 1 0 0 0 0 0 2463 1 3 4 0 0 1 0 0 2 0 0 27 0 0 1 2 2 1 1 0 good
766 1 12 2 0 0 0 1 0 0 1155 0 4 3 0 0 1 0 1 2 1 0 40 1 0 1 2 1 1 0 0 good
767 0 30 2 0 0 1 0 0 0 3108 0 1 2 1 0 0 0 0 3 0 0 31 0 0 1 1 1 1 0 0 bad
768 3 10 2 0 1 0 0 0 0 2901 4 1 1 0 0 0 0 0 3 1 0 31 0 1 0 1 2 1 0 0 good
769 1 12 4 0 0 1 0 0 0 3617 0 4 1 0 1 0 0 0 3 0 0 28 0 1 0 3 2 1 1 0 good
770 3 12 4 0 0 0 1 0 0 1655 0 4 2 0 1 0 0 0 3 1 0 63 0 0 1 2 1 1 1 0 good
771 0 24 2 0 1 0 0 0 0 2812 4 4 2 0 0 0 0 0 3 1 0 26 0 1 0 1 2 1 0 0 good
772 0 36 4 0 0 0 0 1 0 8065 0 2 3 0 0 0 0 0 1 0 1 25 0 0 1 2 3 1 1 0 bad
773 3 21 4 0 1 0 0 0 0 3275 0 4 1 0 1 0 0 0 3 0 0 36 0 0 1 1 3 1 1 0 good
774 3 24 4 0 0 0 1 0 0 2223 1 4 4 0 1 0 0 0 3 0 0 52 1 0 1 2 2 1 0 0 good
775 2 12 4 1 0 0 0 0 0 1480 2 0 2 0 1 0 0 0 3 0 1 66 1 0 0 3 0 1 0 0 good
776 0 24 2 1 0 0 0 0 0 1371 4 2 4 0 0 0 0 0 3 1 0 25 0 1 0 1 2 1 0 0 bad
777 3 36 4 1 0 0 0 0 0 3535 0 3 4 0 1 0 0 0 3 0 0 37 0 0 1 2 2 1 1 0 good
778 0 18 2 0 0 0 1 0 0 3509 0 3 4 0 0 0 0 1 0 1 0 25 0 0 1 1 2 1 0 0 good
779 3 36 4 0 1 0 0 0 0 5711 3 4 4 0 1 0 0 0 1 0 0 38 0 0 1 2 3 1 1 0 good
780 1 18 2 0 0 0 0 0 0 3872 0 0 2 0 0 0 0 0 3 0 0 67 0 0 1 1 2 1 1 0 good
781 1 39 4 0 0 0 1 0 0 4933 0 3 2 0 1 0 0 1 1 1 0 25 0 0 1 2 2 1 0 0 bad
782 3 24 4 1 0 0 0 0 0 1940 3 4 4 0 1 0 0 0 3 1 0 60 0 0 1 1 2 1 1 0 good
783 1 12 0 0 0 0 0 0 0 1410 0 2 2 0 1 0 0 0 1 1 0 31 0 0 1 1 1 1 1 0 good
784 1 12 2 1 0 0 0 0 0 836 1 1 4 0 0 0 0 0 1 0 0 23 1 0 1 1 1 1 0 0 bad
785 1 20 2 0 1 0 0 0 0 6468 4 0 1 1 0 0 0 0 3 1 0 60 0 0 1 1 3 1 1 0 good
786 1 18 2 0 0 0 0 0 1 1941 3 2 4 0 1 0 0 0 1 0 0 35 0 0 1 1 1 1 1 0 good
787 3 22 2 0 0 0 1 0 0 2675 2 4 3 0 1 0 0 0 3 0 0 40 0 0 1 1 2 1 0 0 good
788 3 48 4 0 1 0 0 0 0 2751 4 4 4 0 1 0 0 0 2 0 0 38 0 0 1 2 2 2 1 0 good
789 1 48 3 0 0 0 0 1 0 6224 0 4 4 0 1 0 0 0 3 0 1 50 0 0 0 1 2 1 0 0 bad
790 0 40 4 0 0 0 0 1 0 5998 0 2 4 0 1 0 0 0 2 0 1 27 1 0 1 1 2 1 1 0 bad
791 1 21 2 0 0 0 0 0 1 1188 0 4 2 0 0 0 0 0 3 0 0 39 0 0 1 1 2 2 0 0 bad
792 3 24 2 0 1 0 0 0 0 6313 4 4 3 0 1 0 0 0 3 0 0 41 0 0 1 1 3 2 1 0 good
793 3 6 4 0 0 1 0 0 0 1221 4 2 1 0 0 1 0 0 1 0 0 27 0 0 1 2 2 1 0 0 good
794 2 24 2 0 0 1 0 0 0 2892 0 4 3 1 0 0 0 0 3 0 1 51 0 0 0 1 2 1 0 0 good
795 3 24 2 0 0 1 0 0 0 3062 2 4 4 0 1 0 0 0 2 0 1 32 0 1 0 1 2 1 1 0 good
796 3 9 2 0 0 1 0 0 0 2301 1 1 2 0 0 0 0 0 3 0 0 22 0 1 0 1 2 1 0 0 good
797 0 18 2 0 1 0 0 0 0 7511 4 4 1 0 1 0 0 0 3 0 0 51 0 0 0 1 2 2 1 0 bad
798 3 12 4 0 0 1 0 0 0 1258 0 1 2 0 0 0 0 0 3 0 0 22 0 1 0 2 1 1 0 0 good
799 3 24 3 1 0 0 0 0 0 717 4 4 4 0 0 1 0 0 3 0 0 54 0 0 1 2 2 1 1 0 good
800 1 9 2 1 0 0 0 0 0 1549 4 1 4 0 1 0 0 0 1 1 0 35 0 0 1 1 0 1 0 0 good
801 3 24 4 0 0 0 0 1 0 1597 0 4 4 0 1 0 0 0 3 0 1 54 0 0 0 2 2 2 0 0 good
802 1 18 4 0 0 0 1 0 0 1795 0 4 3 0 0 0 0 1 3 1 0 48 1 1 0 2 1 1 1 0 good
803 0 20 4 0 0 1 0 0 0 4272 0 4 1 0 0 0 0 0 3 0 0 24 0 0 1 2 2 1 0 0 good
804 3 12 4 0 0 0 1 0 0 976 4 4 4 0 1 0 0 0 3 0 0 35 0 0 1 2 2 1 0 0 good
805 1 12 2 1 0 0 0 0 0 7472 4 0 1 0 0 0 0 0 1 1 0 24 0 1 0 1 0 1 0 0 good
806 0 36 2 1 0 0 0 0 0 9271 0 3 2 0 1 0 0 0 0 0 0 24 0 0 1 1 2 1 1 0 bad
807 1 6 2 0 0 0 1 0 0 590 0 1 3 0 0 1 0 0 2 1 0 26 0 0 1 1 1 1 0 1 good
808 3 12 4 0 0 0 1 0 0 930 4 4 4 0 1 0 0 0 3 1 0 65 0 0 1 4 2 1 0 0 good
809 1 42 1 0 1 0 0 0 0 9283 0 0 1 0 1 0 0 0 1 0 1 55 1 0 0 1 3 1 1 0 good
810 1 15 0 1 0 0 0 0 0 1778 0 1 2 0 0 0 0 0 0 1 0 26 0 1 0 2 0 1 0 0 bad
811 1 8 2 0 0 0 0 0 1 907 0 1 3 0 0 1 0 0 1 1 0 26 0 0 1 1 2 1 1 0 good
812 1 6 2 0 0 0 1 0 0 484 0 3 3 0 0 1 0 1 2 1 0 28 1 0 1 1 1 1 0 0 good
813 0 36 4 0 1 0 0 0 0 9629 0 3 4 0 1 0 0 0 3 0 0 24 0 0 1 2 2 1 1 0 bad
814 0 48 2 0 0 0 0 0 0 3051 0 2 3 0 1 0 0 0 3 0 0 54 0 0 1 1 2 1 0 0 bad
815 0 48 2 1 0 0 0 0 0 3931 0 3 4 0 1 0 0 0 3 0 1 46 0 0 0 1 2 2 0 0 bad
816 1 36 3 1 0 0 0 0 0 7432 0 2 2 0 0 0 0 0 1 0 0 54 0 1 0 1 2 1 0 0 good
817 3 6 2 0 0 0 0 0 0 1338 2 2 1 1 0 0 0 0 3 1 0 62 0 0 1 1 2 1 0 0 good
818 3 6 4 0 0 0 1 0 0 1554 0 3 1 0 0 0 0 0 1 0 0 24 0 1 0 2 2 1 1 0 good
819 0 36 2 0 0 0 0 0 0 15857 0 0 2 1 0 0 1 0 2 0 0 43 0 0 1 1 3 1 0 0 good
820 0 18 2 0 0 0 1 0 0 1345 0 2 4 0 0 1 0 0 2 1 0 26 1 0 1 1 2 1 0 0 bad
821 3 12 2 1 0 0 0 0 0 1101 0 2 3 0 0 1 0 0 1 1 0 27 0 0 1 2 2 1 1 0 good
822 2 12 2 0 0 0 1 0 0 3016 0 2 3 0 0 1 0 0 0 0 0 24 0 0 1 1 2 1 0 0 good
823 0 36 2 0 0 1 0 0 0 2712 0 4 2 0 1 0 0 0 1 0 0 41 1 0 1 1 2 2 0 0 bad
824 0 8 4 1 0 0 0 0 0 731 0 4 4 0 1 0 0 0 3 1 0 47 0 0 1 2 1 1 0 0 good
825 3 18 4 0 0 1 0 0 0 3780 0 1 3 1 0 0 0 0 1 0 0 35 0 0 1 2 3 1 1 0 good
826 0 21 4 1 0 0 0 0 0 1602 0 4 4 0 0 1 0 0 2 0 0 30 0 0 1 2 2 1 1 0 good
827 0 18 4 1 0 0 0 0 0 3966 0 4 1 0 0 0 0 0 3 1 0 33 1 1 0 3 2 1 1 0 bad
828 3 18 0 0 0 0 0 0 1 4165 0 2 2 0 1 0 0 0 1 0 0 36 1 0 1 2 2 2 0 0 bad
829 0 36 2 0 1 0 0 0 0 8335 4 4 3 0 1 0 0 0 3 0 1 47 0 0 0 1 2 1 0 0 bad
830 1 48 3 0 0 0 0 0 1 6681 4 2 4 0 1 0 0 0 3 0 1 38 0 0 0 1 2 2 1 0 good
831 3 24 3 0 0 0 0 0 1 2375 2 2 4 0 1 0 0 0 1 0 0 44 0 0 1 2 2 2 1 0 good
832 0 18 2 1 0 0 0 0 0 1216 0 1 4 0 0 0 0 0 2 0 0 23 0 1 0 1 2 1 1 0 bad
833 0 45 0 0 0 0 0 0 1 11816 0 4 2 0 1 0 0 0 3 0 0 29 0 1 0 2 2 1 0 0 bad
834 1 24 2 0 0 0 1 0 0 5084 4 4 2 0 0 0 0 0 3 0 0 42 0 0 1 1 2 1 1 0 good
835 2 15 2 0 0 0 1 0 0 2327 0 1 2 0 0 0 0 0 2 1 0 25 0 0 1 1 1 1 0 0 bad
836 0 12 0 1 0 0 0 0 0 1082 0 2 4 0 1 0 0 0 3 0 0 48 1 0 1 2 2 1 0 0 bad
837 3 12 2 0 0 0 1 0 0 886 4 2 4 0 0 0 0 0 1 0 0 21 0 0 1 1 2 1 0 0 good
838 3 4 2 0 0 1 0 0 0 601 0 1 1 0 0 0 0 0 2 1 0 23 0 1 0 1 1 2 0 0 good
839 0 24 4 0 1 0 0 0 0 2957 0 4 4 0 1 0 0 0 3 0 0 63 0 0 1 2 2 1 1 0 good
840 3 24 4 0 0 0 1 0 0 2611 0 4 4 0 0 1 1 0 2 1 0 46 0 0 1 2 2 1 0 0 good
841 0 36 2 0 0 1 0 0 0 5179 0 3 4 0 1 0 0 0 1 0 0 29 0 0 1 1 2 1 0 0 bad
842 3 21 3 0 1 0 0 0 0 2993 0 2 3 0 1 0 0 0 1 1 0 28 1 0 1 2 1 1 0 0 good
843 3 18 2 0 0 0 0 0 0 1943 0 1 4 0 0 0 0 0 3 1 0 23 0 0 1 1 2 1 0 0 bad
844 3 24 1 0 0 0 0 0 1 1559 0 3 4 0 1 0 0 0 3 0 0 50 1 0 1 1 2 1 1 0 good
845 3 18 2 0 0 1 0 0 0 3422 0 4 4 0 1 0 0 0 3 0 0 47 1 0 1 3 2 2 1 0 good
846 1 21 2 0 0 1 0 0 0 3976 4 3 2 0 1 0 0 0 2 0 0 35 0 0 1 1 2 1 1 0 good
847 3 18 2 1 0 0 0 0 0 6761 4 2 2 0 1 0 0 0 3 0 0 68 0 1 0 2 2 1 0 0 bad
848 3 24 2 1 0 0 0 0 0 1249 0 1 4 0 0 1 0 0 1 1 0 28 0 0 1 1 2 1 0 0 good
849 0 9 2 0 0 0 1 0 0 1364 0 3 3 0 1 0 0 0 3 1 0 59 0 0 1 1 2 1 0 0 good
850 0 12 2 0 0 0 1 0 0 709 0 4 4 0 1 0 0 0 3 1 0 57 1 0 1 1 1 1 0 0 bad
851 0 20 4 1 0 0 0 0 0 2235 0 2 4 0 0 1 0 1 1 0 0 33 1 1 0 2 2 1 0 1 bad
852 3 24 4 0 1 0 0 0 0 4042 4 3 3 0 1 0 0 0 3 0 0 43 0 0 1 2 2 1 1 0 good
853 3 15 4 0 0 0 1 0 0 1471 0 2 4 0 1 0 0 0 3 0 1 35 0 0 0 2 2 1 1 0 good
854 0 18 1 1 0 0 0 0 0 1442 0 3 4 0 1 0 0 0 3 0 1 32 0 0 0 2 1 2 0 0 bad
855 3 36 3 1 0 0 0 0 0 10875 0 4 2 0 1 0 0 0 1 0 0 45 0 0 1 2 2 2 1 0 good
856 3 24 2 1 0 0 0 0 0 1474 1 1 4 0 0 1 0 0 2 1 0 33 0 0 1 1 2 1 1 0 good
857 3 10 2 0 0 0 0 0 0 894 4 3 4 0 0 0 0 0 2 0 0 40 0 0 1 1 2 1 1 0 good
858 3 15 4 0 0 1 0 0 0 3343 0 2 4 0 1 0 0 0 1 0 1 28 0 0 0 1 2 1 1 0 good
859 0 15 2 1 0 0 0 0 0 3959 0 2 3 0 0 0 0 0 1 0 0 29 0 0 1 1 2 1 1 0 bad
860 3 9 2 1 0 0 0 0 0 3577 1 2 1 0 1 0 0 1 1 1 0 26 0 1 0 1 2 2 0 1 good
861 3 24 4 0 1 0 0 0 0 5804 3 2 4 0 1 0 0 0 1 1 0 27 0 0 1 2 2 1 0 0 good
862 3 18 3 0 0 0 0 0 1 2169 0 2 4 0 0 1 0 0 1 0 0 28 0 0 1 1 2 1 1 0 bad
863 0 24 2 0 0 0 1 0 0 2439 0 1 4 0 0 0 0 0 3 1 0 35 0 0 1 1 2 1 1 0 bad
864 3 27 4 0 0 1 0 0 0 4526 3 1 4 0 1 0 0 0 1 1 0 32 1 0 1 2 1 2 1 0 good
865 3 10 2 0 0 1 0 0 0 2210 0 2 2 0 1 0 0 0 1 1 0 25 1 1 0 1 1 1 0 0 bad
866 3 15 2 0 0 1 0 0 0 2221 2 2 2 0 0 0 0 0 3 0 0 20 0 1 0 1 2 1 0 0 good
867 0 18 2 0 0 0 1 0 0 2389 0 1 4 0 0 0 0 0 0 0 0 27 1 0 1 1 2 1 0 0 good
868 3 12 4 0 0 1 0 0 0 3331 0 4 2 0 1 0 0 0 3 0 0 42 1 0 1 1 2 1 0 0 good
869 3 36 2 0 0 0 0 0 1 7409 4 4 3 0 1 0 0 0 1 0 0 37 0 0 1 2 2 1 0 0 good
870 0 12 2 0 0 1 0 0 0 652 0 4 4 0 0 0 0 0 3 0 0 24 0 1 0 1 2 1 0 0 good
871 3 36 3 0 0 1 0 0 0 7678 2 3 2 0 0 0 0 0 3 0 0 40 0 0 1 2 2 1 1 0 good
872 2 6 4 1 0 0 0 0 0 1343 0 4 1 0 1 0 0 0 3 1 0 46 0 0 1 2 2 2 0 1 good
873 0 24 4 0 0 0 0 0 1 1382 1 3 4 0 1 0 0 0 0 1 0 26 0 0 1 2 2 1 1 0 good
874 3 15 2 0 0 0 0 0 0 874 4 1 4 0 0 0 0 0 0 1 0 24 0 0 1 1 2 1 0 0 good
875 0 12 2 0 0 1 0 0 0 3590 0 2 2 0 1 0 1 0 1 0 0 29 0 0 1 1 1 2 0 0 good
876 1 11 4 1 0 0 0 0 0 1322 3 2 4 0 0 0 0 0 3 0 0 40 0 0 1 2 2 1 0 0 good
877 0 18 1 0 0 0 1 0 0 1940 0 1 3 0 1 0 1 0 3 0 1 36 1 0 0 1 3 1 1 0 good
878 3 36 2 0 0 0 1 0 0 3595 0 4 4 0 1 0 0 0 1 0 0 28 0 0 1 1 2 1 0 0 good
879 0 9 2 1 0 0 0 0 0 1422 0 1 3 0 1 0 0 0 1 0 1 27 0 0 0 1 3 1 1 0 bad
880 3 30 4 0 0 0 1 0 0 6742 4 3 2 0 1 0 0 0 2 0 0 36 0 0 1 2 2 1 0 0 good
881 3 24 2 0 1 0 0 0 0 7814 0 3 3 0 1 0 0 0 2 0 0 38 0 0 1 1 3 1 1 0 good
882 3 24 2 0 1 0 0 0 0 9277 4 2 2 1 0 0 0 0 3 0 1 48 0 0 0 1 2 1 1 0 good
883 1 30 4 1 0 0 0 0 0 2181 4 4 4 0 1 0 0 0 3 1 0 36 0 0 1 2 2 1 0 0 good
884 3 18 4 0 0 0 1 0 0 1098 0 0 4 0 0 0 0 0 3 0 0 65 0 0 1 2 0 1 0 0 good
885 1 24 2 0 0 1 0 0 0 4057 0 3 3 1 0 0 0 0 2 0 0 43 0 0 1 1 2 1 1 0 bad
886 0 12 2 0 0 0 0 1 0 795 0 1 4 0 0 0 0 0 3 0 0 53 0 0 1 1 2 1 0 0 bad
887 1 24 4 0 0 0 0 0 1 2825 4 3 4 0 1 0 0 0 2 0 1 34 0 0 1 2 2 2 1 0 good
888 1 48 2 0 0 0 0 0 1 15672 0 2 2 0 1 0 0 0 1 0 0 23 0 0 1 1 2 1 1 0 bad
889 3 36 4 1 0 0 0 0 0 6614 0 4 4 0 1 0 0 0 3 0 0 34 0 0 1 2 3 1 1 0 good
890 3 28 1 0 1 0 0 0 0 7824 4 1 3 0 1 0 0 1 3 1 0 40 1 1 0 2 2 2 1 0 good
891 0 27 4 0 0 0 0 0 1 2442 0 4 4 0 1 0 0 0 3 0 0 43 1 0 1 4 3 2 1 0 good
892 3 15 4 0 0 0 1 0 0 1829 0 4 4 0 1 0 0 0 3 0 0 46 0 0 1 2 2 1 1 0 good
893 0 12 4 1 0 0 0 0 0 2171 0 2 4 0 1 0 0 0 3 0 0 38 1 0 1 2 1 1 0 1 good
894 1 36 4 0 1 0 0 0 0 5800 0 2 3 0 1 0 0 0 3 0 0 34 0 0 1 2 2 1 1 0 good
895 3 18 4 0 0 0 1 0 0 1169 4 2 4 0 1 0 0 0 2 0 0 29 0 0 1 2 2 1 1 0 good
896 3 36 3 0 1 0 0 0 0 8947 4 3 3 0 1 0 0 0 1 0 0 31 1 0 1 1 3 2 1 0 good
897 0 21 2 0 0 0 1 0 0 2606 0 1 4 0 0 0 0 0 3 0 0 28 0 1 0 1 3 1 1 0 good
898 3 12 4 0 0 1 0 0 0 1592 3 3 3 0 0 0 0 0 1 0 0 35 0 0 1 1 2 1 0 1 good
899 3 15 2 0 0 1 0 0 0 2186 4 3 1 0 0 0 0 0 3 1 0 33 1 1 0 1 1 1 0 0 good
900 0 18 2 0 0 1 0 0 0 4153 0 2 2 0 1 0 1 0 2 0 0 42 0 0 1 1 2 1 0 0 bad
901 0 16 4 1 0 0 0 0 0 2625 0 4 2 0 1 0 0 1 3 0 0 43 1 1 0 1 2 1 1 0 bad
902 3 20 4 1 0 0 0 0 0 3485 4 1 2 1 0 0 0 0 3 1 0 44 0 0 1 2 2 1 1 0 good
903 3 36 4 0 1 0 0 0 0 10477 4 4 2 0 1 0 0 0 3 0 1 42 0 0 0 2 2 1 0 0 good
904 3 15 2 0 0 0 1 0 0 1386 4 2 4 0 0 1 0 0 1 1 0 40 0 1 0 1 2 1 1 0 good
905 3 24 2 0 0 0 1 0 0 1278 0 4 4 0 1 0 0 0 0 1 0 36 0 0 1 1 3 1 1 0 good
906 0 12 2 0 0 0 1 0 0 1107 0 2 2 0 1 0 0 0 1 1 0 20 0 1 0 1 3 2 1 0 good
907 0 21 2 1 0 0 0 0 0 3763 4 3 2 0 1 0 1 0 1 1 0 24 0 0 1 1 1 1 0 1 good
908 1 36 2 0 0 0 0 1 0 3711 4 2 2 0 0 1 0 0 1 0 0 27 0 0 1 1 2 1 0 0 good
909 3 15 3 0 1 0 0 0 0 3594 0 1 1 0 0 0 0 0 1 0 0 46 0 0 1 2 1 1 0 0 good
910 1 9 2 1 0 0 0 0 0 3195 4 2 1 0 0 0 0 0 1 1 0 33 0 0 1 1 1 1 0 0 good
911 3 36 3 0 0 0 1 0 0 4454 0 2 4 0 0 0 0 0 3 1 0 34 0 0 1 2 2 1 0 0 good
912 1 24 4 0 0 1 0 0 0 4736 0 1 2 0 0 0 0 0 3 0 0 25 1 0 1 1 1 1 0 0 bad
913 1 30 2 0 0 0 1 0 0 2991 4 4 2 0 0 0 0 0 3 0 0 25 0 0 1 1 2 1 0 0 good
914 3 11 2 0 0 0 0 0 1 2142 3 4 1 1 0 0 0 0 1 1 0 28 0 0 1 1 2 1 1 0 good
915 0 24 1 0 0 0 0 0 1 3161 0 2 4 0 1 0 0 0 1 0 0 31 0 1 0 1 2 1 1 0 bad
916 1 48 0 0 0 0 0 0 0 18424 0 2 1 0 0 0 0 0 1 0 0 32 1 0 1 1 3 1 1 1 bad
917 3 10 2 0 1 0 0 0 0 2848 1 2 1 0 1 0 1 0 1 1 0 32 0 0 1 1 2 2 0 0 good
918 0 6 2 1 0 0 0 0 0 14896 0 4 1 0 1 0 0 0 3 0 1 68 1 0 1 1 3 1 1 0 bad
919 0 24 2 0 0 1 0 0 0 2359 1 0 1 1 0 0 0 0 0 0 0 33 0 0 1 1 2 1 0 0 bad
920 0 24 2 0 0 1 0 0 0 3345 0 4 4 0 1 0 0 0 1 0 0 39 0 1 0 1 3 1 1 0 bad
921 3 18 4 0 0 1 0 0 0 1817 0 2 4 0 0 0 0 0 1 0 1 28 0 0 1 2 2 1 0 0 good
922 3 48 3 0 0 0 1 0 0 12749 2 3 4 0 1 0 0 0 0 0 0 37 0 0 1 1 3 1 1 0 good
923 0 9 2 0 0 0 1 0 0 1366 0 1 3 0 0 0 0 0 3 0 0 22 0 1 0 1 2 1 0 0 bad
924 1 12 2 1 0 0 0 0 0 2002 0 3 3 0 1 0 0 0 3 0 0 30 0 1 0 1 2 2 1 0 good
925 0 24 1 0 0 1 0 0 0 6872 0 1 2 1 0 0 0 0 0 0 0 55 1 0 1 1 2 1 1 0 bad
926 0 12 1 1 0 0 0 0 0 697 0 1 4 0 1 0 0 0 1 0 0 46 1 0 1 2 2 1 1 0 bad
927 0 18 4 0 0 1 0 0 0 1049 0 1 4 0 0 0 0 0 3 0 0 21 0 1 0 1 2 1 0 0 good
928 0 48 2 0 1 0 0 0 0 10297 0 3 4 0 1 0 0 0 3 0 1 39 1 0 0 3 2 2 1 0 bad
929 3 30 2 0 0 0 1 0 0 1867 4 4 4 0 1 0 0 0 3 0 0 58 0 0 1 1 2 1 1 0 good
930 0 12 3 1 0 0 0 0 0 1344 0 2 4 0 1 0 0 0 1 1 0 43 0 0 1 2 1 2 0 0 good
931 0 24 2 0 0 1 0 0 0 1747 0 1 4 0 1 0 1 0 0 0 0 24 0 0 1 1 1 1 0 1 good
932 1 9 2 0 0 0 1 0 0 1670 0 1 4 0 0 0 0 0 1 0 0 22 0 0 1 1 2 1 1 0 bad
933 3 9 4 1 0 0 0 0 0 1224 0 2 3 0 1 0 0 0 0 1 0 30 0 0 1 2 2 1 0 0 good
934 3 12 4 0 0 0 1 0 0 522 2 4 4 0 1 0 0 0 3 0 0 42 0 0 1 2 2 2 1 0 good
935 0 12 2 0 0 0 1 0 0 1498 0 2 4 0 0 0 0 0 0 0 0 23 1 0 1 1 2 1 0 0 good
936 1 30 3 0 0 0 1 0 0 1919 1 1 4 0 1 0 0 0 2 0 1 30 1 0 1 2 3 1 0 0 bad
937 2 9 2 0 0 0 1 0 0 745 0 2 3 0 0 0 0 0 1 1 0 28 0 0 1 1 1 1 0 0 bad
938 1 6 2 0 0 0 1 0 0 2063 0 1 4 0 0 1 0 0 2 0 0 30 0 1 0 1 3 1 1 0 good
939 1 60 2 0 0 0 0 1 0 6288 0 2 4 0 1 0 0 0 3 0 1 42 0 0 0 1 2 1 0 0 bad
940 3 24 4 0 1 0 0 0 0 6842 4 2 2 0 1 0 0 0 3 0 0 46 0 0 1 2 3 2 1 0 good
941 3 12 2 1 0 0 0 0 0 3527 4 1 2 0 1 0 0 0 2 0 0 45 0 0 1 1 3 2 1 0 good
942 3 10 2 1 0 0 0 0 0 1546 0 2 3 0 1 0 0 0 1 1 0 31 0 0 1 1 1 2 0 1 good
943 3 24 2 0 0 1 0 0 0 929 4 3 4 0 1 0 0 0 1 0 0 31 1 0 1 1 2 1 1 0 good
944 3 4 4 1 0 0 0 0 0 1455 0 3 2 0 1 0 0 0 0 1 0 42 0 0 1 3 1 2 0 0 good
945 0 15 2 0 0 1 0 0 0 1845 0 1 4 0 0 0 0 1 0 0 0 46 0 1 0 1 2 1 0 0 good
946 1 48 0 1 0 0 0 0 0 8358 2 1 1 0 0 0 0 0 0 0 0 30 0 0 1 2 2 1 0 0 good
947 0 24 1 0 0 1 0 0 0 3349 2 1 4 0 1 0 0 0 3 0 1 30 0 0 0 1 2 2 1 0 bad
948 3 12 2 1 0 0 0 0 0 2859 4 0 4 0 1 0 0 0 3 0 1 38 0 0 1 1 3 1 1 0 good
949 3 18 2 0 0 1 0 0 0 1533 0 1 4 0 0 1 1 0 0 0 0 43 0 0 1 1 1 2 0 0 bad
950 3 24 2 0 0 0 1 0 0 3621 1 4 2 0 1 0 0 0 3 0 0 31 0 0 1 2 2 1 0 0 bad
951 1 18 4 0 0 0 0 0 1 3590 0 0 3 0 0 1 0 0 2 0 0 40 0 0 1 3 0 2 1 0 good
952 0 36 3 0 0 0 0 0 1 2145 0 3 2 0 1 0 0 0 0 0 0 24 0 0 1 2 2 1 1 0 bad
953 1 24 2 0 1 0 0 0 0 4113 2 1 3 0 0 0 0 0 3 0 0 28 0 1 0 1 2 1 0 0 bad
954 3 36 2 0 0 1 0 0 0 10974 0 0 4 0 0 0 0 0 1 0 0 26 0 0 1 2 3 1 1 0 bad
955 0 12 2 1 0 0 0 0 0 1893 0 2 4 0 0 0 0 1 3 0 0 29 0 0 1 1 2 1 1 0 good
956 0 24 4 0 0 0 1 0 0 1231 3 4 4 0 0 0 0 0 3 0 0 57 0 1 0 2 3 1 1 0 good
957 2 30 4 0 0 0 1 0 0 3656 4 4 4 0 1 0 0 0 3 0 0 49 1 0 1 2 1 1 0 0 good
958 1 9 4 0 0 0 1 0 0 1154 0 4 2 0 1 0 0 0 3 1 0 37 0 0 1 3 1 1 0 0 good
959 0 28 2 1 0 0 0 0 0 4006 0 2 3 0 1 0 0 0 1 0 0 45 0 0 1 1 1 1 0 0 bad
960 1 24 2 0 0 1 0 0 0 3069 1 4 4 0 1 0 0 0 3 0 1 30 0 0 0 1 2 1 0 0 good
961 3 6 4 0 0 0 1 0 0 1740 0 4 2 0 0 1 0 0 1 1 0 30 0 1 0 2 2 1 0 0 good
962 1 21 3 1 0 0 0 0 0 2353 0 2 1 1 0 0 0 0 3 0 0 47 0 0 1 2 2 1 0 0 good
963 3 15 2 1 0 0 0 0 0 3556 4 2 3 0 1 0 0 0 1 0 1 29 0 0 1 1 2 1 0 0 good
964 3 24 2 0 0 0 1 0 0 2397 2 4 3 0 1 0 0 0 1 0 0 35 1 0 1 2 2 1 1 0 bad
965 1 6 2 0 0 0 0 0 0 454 0 1 3 0 0 1 0 0 0 0 0 22 0 0 1 1 1 1 0 0 good
966 1 30 2 0 0 0 1 0 0 1715 4 2 4 0 0 0 0 0 0 0 0 26 0 0 1 1 2 1 0 0 good
967 1 27 4 0 0 0 1 0 0 2520 2 2 4 0 1 0 0 0 1 0 0 23 0 0 1 2 1 1 0 0 bad
968 3 15 2 0 0 0 1 0 0 3568 0 4 4 0 0 0 0 0 1 0 0 54 1 1 0 1 3 1 1 0 good
969 3 42 2 0 0 0 1 0 0 7166 4 3 2 0 0 1 0 0 3 0 0 29 0 1 0 1 2 1 1 0 good
970 0 11 4 1 0 0 0 0 0 3939 0 2 1 0 1 0 0 0 1 1 0 40 0 0 1 2 1 2 0 0 good
971 1 15 2 0 0 0 0 0 0 1514 1 2 4 0 1 0 0 1 1 1 0 22 0 0 1 1 2 1 0 0 good
972 3 24 2 1 0 0 0 0 0 7393 0 2 1 0 1 0 0 0 3 0 0 43 0 0 1 1 1 2 0 0 good
973 0 24 1 1 0 0 0 0 0 1193 0 0 1 0 0 0 1 0 3 0 1 29 0 1 0 2 0 1 0 0 bad
974 0 60 2 0 0 0 0 0 1 7297 0 4 4 0 1 0 1 0 3 0 1 36 0 1 0 1 2 1 0 0 bad
975 3 30 4 0 0 0 1 0 0 2831 0 2 4 0 0 0 0 0 1 0 0 33 0 0 1 1 2 1 1 0 good
976 2 24 2 0 0 0 1 0 0 1258 2 2 3 0 0 0 0 0 2 0 0 57 0 0 1 1 1 1 0 0 good
977 1 6 2 0 0 0 1 0 0 753 0 2 2 0 0 0 0 1 2 1 0 64 0 0 1 1 2 1 0 0 good
978 1 18 3 0 0 0 0 0 1 2427 4 4 4 0 1 0 0 0 1 0 0 42 0 0 1 2 2 1 0 0 good
979 3 24 3 1 0 0 0 0 0 2538 0 4 4 0 1 0 0 0 3 0 0 47 0 0 1 2 1 2 0 0 bad
980 1 15 1 1 0 0 0 0 0 1264 1 2 2 0 0 1 0 0 1 0 0 25 0 1 0 1 2 1 0 0 bad
981 1 30 4 0 0 1 0 0 0 8386 0 3 2 0 1 0 0 0 1 0 0 49 0 0 1 1 2 1 0 0 bad
982 3 48 2 0 0 0 0 0 1 4844 0 0 3 0 1 0 0 0 1 0 0 33 1 1 0 1 3 1 1 0 bad
983 2 21 2 1 0 0 0 0 0 2923 1 2 1 0 0 0 0 0 0 0 0 28 1 0 1 1 3 1 1 0 good
984 0 36 2 0 1 0 0 0 0 8229 0 2 2 0 1 0 0 0 1 0 0 26 0 0 1 1 2 2 0 0 bad
985 3 24 4 0 0 1 0 0 0 2028 0 3 2 0 1 0 0 0 1 0 0 30 0 0 1 2 1 1 0 0 good
986 0 15 4 0 0 1 0 0 0 1433 0 2 4 0 0 0 0 0 2 0 0 25 0 1 0 2 2 1 0 0 good
987 2 42 0 0 0 0 0 0 1 6289 0 1 2 1 0 0 0 0 0 0 0 33 0 0 1 2 2 1 0 0 good
988 3 13 2 0 0 0 1 0 0 1409 1 0 2 0 0 0 0 0 3 1 0 64 0 0 1 1 2 1 0 0 good
989 0 24 2 0 1 0 0 0 0 6579 0 0 4 0 1 0 0 0 1 0 1 29 0 0 0 1 3 1 1 0 good
990 1 24 4 0 0 0 1 0 0 1743 0 4 4 0 1 0 0 0 1 0 0 48 0 0 1 2 1 1 0 0 good
991 3 12 4 0 0 0 0 1 0 3565 4 1 2 0 1 0 0 0 0 0 0 37 0 0 1 2 1 2 0 0 good
992 3 15 1 0 0 0 1 0 0 1569 1 4 4 0 1 0 0 0 3 0 0 34 1 0 1 1 1 2 0 0 good
993 0 18 2 0 0 0 1 0 0 1936 4 3 2 0 0 1 0 0 3 0 0 23 0 1 0 2 1 1 0 0 good
994 0 36 2 0 0 1 0 0 0 3959 0 0 4 0 1 0 0 0 2 0 0 30 0 0 1 1 3 1 1 0 good
995 3 12 2 1 0 0 0 0 0 2390 4 4 4 0 1 0 0 0 2 0 0 50 0 0 1 1 2 1 1 0 good
996 3 12 2 0 0 1 0 0 0 1736 0 3 3 0 0 0 0 0 3 1 0 31 0 0 1 1 1 1 0 0 good
997 0 30 2 0 1 0 0 0 0 3857 0 2 4 1 0 0 0 0 3 0 0 40 0 0 1 1 3 1 1 0 good
998 3 12 2 0 0 0 1 0 0 804 0 4 4 0 1 0 0 0 3 0 0 38 0 0 1 1 2 1 0 0 good
999 0 45 2 0 0 0 1 0 0 1845 0 2 4 0 1 0 0 0 3 0 1 23 0 0 0 1 2 1 1 0 bad
1000 1 45 4 0 1 0 0 0 0 4576 1 0 3 0 1 0 0 0 3 0 0 27 0 0 1 1 2 1 0 0 good

As already said, we first make the whole procedure on one model before using the caret package.
We choose to present in details the CART model tuned and pruned.
We have already built our 10 different training set and we will use them to make the predictions and calculate the accuracy of our 10 testing set.

CART Model

# creating an empty vector to store the accuracy of each iteration
acc.cart.cv <- numeric(10) 

for (i in 1:10){
  cart.cv <- rpart(response~., data = train.list[[i]]) # original tree
  
  # Tree to be prune at: 
  cp.pruned <- cart.cv$cptable[(
    min(which(cart.cv$cptable[, ncol(cart.cv$cptable) - 1] < 
                cart.cv$cptable[nrow(cart.cv$cptable), 
                                   ncol(cart.cv$cptable)] + 
                cart.cv$cptable[nrow(cart.cv$cptable), 
                                   ncol(cart.cv$cptable) - 1]))), 1] 
  
  # the pruned tree for predictions
  cart.pruned.cv <- prune(cart.cv, cp = cp.pruned) 
  
  # making the predictions
  cart.pred.cv <- predict(cart.pruned.cv, newdata = test.list[[i]], type = "class") 
  
  # the confusion matrix
  tab.cart.cv <- table(test.list[[i]]$response, cart.pred.cv) 
  
  # the final accuracy
  acc.cart.cv[i] <- accuracy(tab.cart.cv) 
}

We can now extract what we are interested in: informations about the accuracy.

acc.cart.cv # the accuracies obtained on each testing set
##  [1] 0.84 0.70 0.75 0.79 0.77 0.66 0.68 0.74 0.76 0.65
mean(acc.cart.cv) # the average accuracy
## [1] 0.734
sd(acc.cart.cv) # the standard deviation of the accuracies
## [1] 0.0607728

Here, we have an average accuracy for the CART model which is equal to 0.734 and that has a standard deviation of 0.0607728. We see that, using a cross validation, the accuracy obtained differ from the one using only the testing/training set approach. However, it is of course still really close to the previous one and within a range of one standard deviation, which was also completely predictable.
Now that we have seen in detail how the cross validation works thanks to this hand-made code, we can prefer to use the caret package that will do the work for us.
Defining train_control, we will be able to use the same split between testing and training set for all the models. We will thus be able to conclude that differences in the accuracy will indeed be the consequence of the predictive capabilities of the model, and not from a “bad” repartition of the instances in the two sets.
Let’s run the cross validation on each model that we previously tuned.

train_control <- trainControl(method = "cv", number = 10)

rpart_cv <- train(form = response~ .,
                 data = cred,
                 trControl = train_control,
                 method = "rpart1SE")

nnet_cv <- train(form = response~ .,
                 data = cred,
                 trControl = train_control,
                 tuneGrid = data.frame(size = nnet_fit$results[which.max(nnet_fit$results$Accuracy),]$size, decay = nnet_fit$results[which.max(nnet_fit$results$Accuracy),]$decay),
                 method = "nnet")
## # weights:  769
## initial  value 2302.323514 
## iter  10 value 552.484828
## iter  20 value 551.477368
## iter  30 value 547.003267
## iter  40 value 536.299861
## iter  50 value 508.988472
## iter  60 value 499.649291
## iter  70 value 480.231509
## iter  80 value 459.571874
## iter  90 value 451.807598
## iter 100 value 446.716653
## final  value 446.716653 
## stopped after 100 iterations
## # weights:  769
## initial  value 1299.126338 
## iter  10 value 573.086805
## iter  20 value 556.833479
## iter  30 value 546.253160
## iter  40 value 531.540364
## iter  50 value 505.114412
## iter  60 value 469.133414
## iter  70 value 464.984674
## iter  80 value 458.324219
## iter  90 value 454.604972
## iter 100 value 453.115679
## final  value 453.115679 
## stopped after 100 iterations
## # weights:  769
## initial  value 796.685409 
## iter  10 value 550.654967
## iter  20 value 543.518834
## iter  30 value 527.400615
## iter  40 value 519.309681
## iter  50 value 469.399578
## iter  60 value 466.900748
## iter  70 value 459.602883
## iter  80 value 457.688642
## iter  90 value 454.692743
## iter 100 value 450.446120
## final  value 450.446120 
## stopped after 100 iterations
## # weights:  769
## initial  value 808.701338 
## iter  10 value 550.502077
## iter  20 value 548.431925
## iter  30 value 545.188449
## iter  40 value 538.503086
## iter  50 value 512.125076
## iter  60 value 469.993680
## iter  70 value 459.457539
## iter  80 value 456.153447
## iter  90 value 450.079226
## iter 100 value 447.794138
## final  value 447.794138 
## stopped after 100 iterations
## # weights:  769
## initial  value 2668.857465 
## iter  10 value 577.838676
## iter  20 value 543.971112
## iter  30 value 534.085163
## iter  40 value 505.243159
## iter  50 value 499.079590
## iter  60 value 478.923942
## iter  70 value 458.520500
## iter  80 value 454.125241
## iter  90 value 451.449131
## iter 100 value 449.337514
## final  value 449.337514 
## stopped after 100 iterations
## # weights:  769
## initial  value 1377.803089 
## iter  10 value 551.456349
## iter  20 value 535.135269
## iter  30 value 500.495838
## iter  40 value 480.883781
## iter  50 value 459.993332
## iter  60 value 453.146587
## iter  70 value 450.318464
## iter  80 value 449.454106
## iter  90 value 447.079427
## iter 100 value 445.009700
## final  value 445.009700 
## stopped after 100 iterations
## # weights:  769
## initial  value 1611.324664 
## iter  10 value 553.129299
## iter  20 value 550.555637
## iter  30 value 531.226539
## iter  40 value 523.602558
## iter  50 value 497.186582
## iter  60 value 484.177951
## iter  70 value 465.634266
## iter  80 value 448.915884
## iter  90 value 444.012504
## iter 100 value 440.592384
## final  value 440.592384 
## stopped after 100 iterations
## # weights:  769
## initial  value 1701.263010 
## iter  10 value 552.413146
## iter  20 value 549.967008
## iter  30 value 549.821336
## iter  40 value 543.921188
## iter  50 value 536.725537
## iter  60 value 505.487795
## iter  70 value 483.848980
## iter  80 value 462.792668
## iter  90 value 458.999305
## iter 100 value 451.625805
## final  value 451.625805 
## stopped after 100 iterations
## # weights:  769
## initial  value 1458.478398 
## iter  10 value 559.449973
## iter  20 value 534.178471
## iter  30 value 523.649992
## iter  40 value 486.849511
## iter  50 value 467.575927
## iter  60 value 455.669045
## iter  70 value 451.307231
## iter  80 value 448.306360
## iter  90 value 441.575883
## iter 100 value 434.753091
## final  value 434.753091 
## stopped after 100 iterations
## # weights:  769
## initial  value 815.028304 
## iter  10 value 550.719947
## iter  20 value 536.652688
## iter  30 value 525.971557
## iter  40 value 512.273182
## iter  50 value 495.674456
## iter  60 value 463.860626
## iter  70 value 455.802158
## iter  80 value 448.033308
## iter  90 value 445.331237
## iter 100 value 444.194128
## final  value 444.194128 
## stopped after 100 iterations
## # weights:  769
## initial  value 1053.136471 
## iter  10 value 611.485761
## iter  20 value 606.291094
## iter  30 value 582.763378
## iter  40 value 572.339530
## iter  50 value 531.695081
## iter  60 value 500.048926
## iter  70 value 498.233310
## iter  80 value 497.571264
## iter  90 value 494.577952
## iter 100 value 489.395475
## final  value 489.395475 
## stopped after 100 iterations
svm_cv <- train(form = response~ .,
                 data = cred,
                 trControl = train_control,
                 tuneGrid = data.frame(
                   C = svm_fit$results[which.max(svm_fit$results$Accuracy),]$C),
                 method = "svmRadialCost")

knn_cv <- train(form = response~ .,
                 data = cred,
                 trControl = train_control,
                 tuneGrid = data.frame(
                   k = knn_fit$results[which.max(knn_fit$results$Accuracy),]$k),
                 method = "knn",
                 preProcess = c("center", "scale"))

rf_cv <- train(form = response ~ .,
                 data = cred,
                 method = "rf",
                 ntree = 500, 
                 tuneGrid = data.frame(
                   mtry=rf_fit$results[which.max(rf_fit$results$Accuracy),]$mtry),
                 preProcess = "range",
                 trace = FALSE,
                 trControl = train_control)

cart_cv <- train(form = response~ .,
                 data = cred,
                 trControl = train_control,
                 tuneGrid = data.frame(
                   k = knn_fit$results[which.max(knn_fit$results$Accuracy),]$k),
                 method = "knn",
                 preProcess = c("center", "scale"))
rpart_cv$results$Accuracy
## [1] 0.757
nnet_cv$results$Accuracy
## [1] 0.769
svm_cv$results$Accuracy
## [1] 0.757
knn_cv$results$Accuracy
## [1] 0.719
rf_cv$results$Accuracy
## [1] 0.761
cart_cv$results$Accuracy
## [1] 0.726